Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Transform a CSV into a JSON and vice versa

jsonmatic.com

Post reply on HN
Print view View original

1–10 of 111 posts

Re: Show HN: Transform a CSV into a JSON and vice versa

#2

Post by brundolf »

Slightly OT: I've realized that CSVs are dramatically more information-dense than the equivalent JSON, and actually make a pretty reasonable API response format if your dataset is large and fits into the tabular shape. They can be a fraction of the size, mainly because keys aren't duplicated for every item.

Re: Show HN: Transform a CSV into a JSON and vice versa

#3

Post by sonthonax »

brundolf wrote: post #2

Slightly OT: I've realized that CSVs are dramatically more information-dense than the equivalent JSON, and actually make a pretty reasonable API response format if your dataset is large and fits into the tabular shape. They can be a fraction of the size, mainly because keys aren't duplicated for every item.

Shouldn’t really matter too much if the response is being compressed.

If you’re rendering the table in the DOM, the response size is the least of your issues.

Re: Show HN: Transform a CSV into a JSON and vice versa

#5

Post by OJFord »

brundolf wrote: post #2

Slightly OT: I've realized that CSVs are dramatically more information-dense than the equivalent JSON, and actually make a pretty reasonable API response format if your dataset is large and fits into the tabular shape. They can be a fraction of the size, mainly because keys aren't duplicated for every item.

Yeah, lists of objects are pretty crap, because they're almost always homogeneous but unenforcédly so; it's not just a size issue but a parsing (or not - usage) issue too.

You could approximate CSV in a JSON response like:

    {
      "columns": ["a", ..., "z"],
      "rows": [[1, ..., 26], ..., [11, 2266]]
    }
Or:

    {
      "a": [1, ..., 11],
      ...
      "z": [26, ..., 2266]
    }
which I've never seen, but would save space, and sort of enforced in the sense that if you trust your serialiser for it as much as you trust an equivalent CSV serialiser, it's fine. (But the same argument could be made for more usual JSON object lists. Only arguable difference is that there's more of an assertion to the client that they should be expected to be homogeneous.)

Re: Show HN: Transform a CSV into a JSON and vice versa

#7

Post by cowsandmilk »

OJFord wrote: post #5
brundolf wrote: post #2

Slightly OT: I've realized that CSVs are dramatically more information-dense than the equivalent JSON, and actually make a pretty reasonable API response format if your dataset is large and fits into the tabular shape. They can be a fraction of the size, mainly because keys aren't duplicated for every item.

Yeah, lists of objects are pretty crap, because they're almost always homogeneous but unenforcédly so; it's not just a size issue but a parsing (or not - usage) issue too. You could approximate CSV in a JSON response like: { "columns": ["a", ..., "z"], "rows": [[1, ..., 26], ..., [11, 2266]] } Or: { "a": [1, ..., 11], ... "z": [26, ..., 2266] } which I've never seen, but would save space, and sort of enforced in the…

I’ll note, your second item here is a structure of arrays, which is often higher performance in practice when you are only interested in certain portions of the data.

For this reason, your second structure is how I serialize code I’m interacting with in C.

Re: Show HN: Transform a CSV into a JSON and vice versa

#8

Post by dmw_ng »

cowsandmilk wrote: post #7
OJFord wrote: post #5

Earlier quoted context omitted.

Yeah, lists of objects are pretty crap, because they're almost always homogeneous but unenforcédly so; it's not just a size issue but a parsing (or not - usage) issue too. You could approximate CSV in a JSON response like: { "columns": ["a", ..., "z"], "rows": [[1, ..., 26], ..., [11, 2266]] } Or: { "a": [1, ..., 11], ... "z": [26, ..., 2266] } which I've never seen, but would save space, and sort of enforced in the…

I’ll note, your second item here is a structure of arrays, which is often higher performance in practice when you are only interested in certain portions of the data. For this reason, your second structure is how I serialize code I’m interacting with in C.

Higher performance, much smaller compressed and uncompressed, accepted by a wide range of tools, e.g. Pandas, and often more convenient to parse from statically typed languages.

Re: Show HN: Transform a CSV into a JSON and vice versa

#9

Post by brundolf »

sonthonax wrote: post #3
brundolf wrote: post #2

Slightly OT: I've realized that CSVs are dramatically more information-dense than the equivalent JSON, and actually make a pretty reasonable API response format if your dataset is large and fits into the tabular shape. They can be a fraction of the size, mainly because keys aren't duplicated for every item.

Shouldn’t really matter too much if the response is being compressed. If you’re rendering the table in the DOM, the response size is the least of your issues.

Sometimes you fetch a large dataset and only show one page at a time in the DOM, or render it as a line in a chart or something. At a previous workplace we had CSV responses in the hundreds of megabytes.

Re: Show HN: Transform a CSV into a JSON and vice versa

#10

Post by killingtime74 »

brundolf wrote: post #2

Slightly OT: I've realized that CSVs are dramatically more information-dense than the equivalent JSON, and actually make a pretty reasonable API response format if your dataset is large and fits into the tabular shape. They can be a fraction of the size, mainly because keys aren't duplicated for every item.

It’s because it’s self describing right? If you look at protobuf, thrift, avro those are even denser
Post reply on HN