21–30 of 42 posts
Earlier quoted context omitted.
It's database agnostic this way and every language supports JSON.
Partly to play devil's advocate and partly out of curiosity, do you know of a language with a JSON library but no SQLite library?
Under US law, the federal government can't claim copyright on works produced via tax dollars (makes sense). Since the feds can't require us to provide attribution for all this helpful data, how do we as a community advocate for more open data like this?
I can imagine only 3 reasons for this: (1) they want to make you agree to the Terms & Conditions, (2) it gives a way to choke off a DDOS attack that makes repeated complex queries, or (3) the census bureau wants to track how you are using its service.
The key is indeed easy to get, but I observe that Google would have exactly the same concerns as 1,2,&3 above, yet they somehow manage to stay in business without making their users sign up to do a web search.
The federal government always makes things a little more complicated.
> The response for all queries is formatted as a two dimensional JSON array where the first row provides column names and subsequent rows provide data values. Hmmm... looking at the example of this I can't help but think there has got to be a better way. This is more just a standard CSV (first row is header, all other rows are data). Using JSON for data formatted as such is kind of a waste of JSON. If you pass that e…
This doesn't seem unreasonable. 1) Whether it's user-friendly or not, it's still JSON, which makes JSON-P possible (they support JSON-P). 2) It probably mirrors the way they store their data (in tables, whether SQL or Access or Excel, doesn't really matter). 3) It's more compact than traditional JSON, which means less bandwidth, which means less cost. Keep in mind that this is essentially a not-for-profit API from a…
- I daresay Census is sitting on some _large_ data sets. Fancy data structures are one thing when you want the population of California, another when you're trying to get California, by ethnicity and age group, by zip code.
- Data structure compactness will matter less to HN readers than to someone sitting on the other end of a phone dial-up in Oklahoma.
- I am but an egg but don't fancier data structures require particular decisions for implementation on particular tables? Census has a lot of tables -> a lot of decisions.
- God only knows how many different systems are involved holding all their data. The simpler the data structure, the less they have to get into those -- and / or the simpler a layer they between some mainframe and the API output.
Also, it may not seem very friendly to HN readers. But it is stupid simple, you can _see_ how it is organized, making it accessible to a wider audience. And it's so simple that it can be adopted by any agency publishing tables. Note that there are a lot of agencies with a lot of tables -- something like this has a chance of becoming standard among all of them.
A public agency has special accessibility concerns, and it's just a reality that government agencies have particular legacy technology issues. A lowest common denominator format helps get the data over those obstacles.
Maybe they can do better in places, and they're free to, there is no reason they can't support other formats too. But this will be available for whatever subset isn't served by those extensions.
You need to request a "key" (essentially a user ID) before you can make queries. I can imagine only 3 reasons for this: (1) they want to make you agree to the Terms & Conditions, (2) it gives a way to choke off a DDOS attack that makes repeated complex queries, or (3) the census bureau wants to track how you are using its service. The key is indeed easy to get, but I observe that Google would have exactly the same co…
Earlier quoted context omitted.
This doesn't seem unreasonable. 1) Whether it's user-friendly or not, it's still JSON, which makes JSON-P possible (they support JSON-P). 2) It probably mirrors the way they store their data (in tables, whether SQL or Access or Excel, doesn't really matter). 3) It's more compact than traditional JSON, which means less bandwidth, which means less cost. Keep in mind that this is essentially a not-for-profit API from a…
All good explanations, but I can't help but wish they had chosen a format like: { P0010001 : [ '710231', '4779736', // ... etc. ], NAME : [ 'Alaska', 'Alabama', // ... etc. ], state: [ '02', '01, // ... etc. ], // ... etc. }
Wouldn't this be better served by a downloadable SQLite file rather than a web service?
for you, and your repliers: this data is already available as a downloadable file. Check out ftp://ftp.census.gov/ It's CSV, and it's kinda ugly-looking, but it's available. The 'full' data set is ...rather huge. The full data set for any particular report is rather huge. The advantage of this API, I expect, is it allows people to make quick, machine readable queries for useful subsets of the data. If you want a Java…
It's great to get excited about the data format, but actual use-cases currently seem a little limited to me. Of course this will hopefully change as more and more datasets become available.