Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Internet Object – A JSON alternative data serialization format

internetobject.org

Post reply on HN
Print view View original

21–30 of 83 posts

Re: Internet Object – A JSON alternative data serialization format

#21

Post by lifthrasiir »

nikeee wrote: post #13

Since strings don't need to be quoted, what happens during deserialization if you want the string "T"? Does this lead to the equivalent of the Norway-Problem of YAML [0]? Is the space between the key and the type necessary? If not, how to distinguish between objects and types? Does the validation offer some form of unions or mutual exclusion? [0]: https://hitchdev.com/strictyaml/why/implicit-typing-removed/

It seems to be a typed CSV, so whether `T` is interpreted as a string or a boolean presumably depends on the schema. That sounds slightly better than YAML, though it can easily break when you allow heterogeneous types (say, string or boolean).

Re: Internet Object – A JSON alternative data serialization format

#22

Post by wffurr »

random478101 wrote: post #12

As far I can see "IO" addresses the size issue, which is indeed a compression issue for the most part. For a broader take on an alternative, there is concise encoding Concise Encoding [1][2], which I believe addresses a few more issues with existing encodings (clear spec, schema not an afterthought, native support for a variety of data structures, security, ...). [1] https://concise-encoding.org/ [2] The author gave…

People keep saying “just use gzip and JSON is plenty small” but gzip isn’t free. It takes time and power to do all the compression and decompression. The uncompressed size of the data takes up memory on client and server.

A smaller data format requires less compression time and power and you can fit more of it in memory at either end.

Re: Internet Object – A JSON alternative data serialization format

#23

Post by colejohnson66 »

cookiengineer wrote: post #18
nikeee wrote: post #13

Since strings don't need to be quoted, what happens during deserialization if you want the string "T"? Does this lead to the equivalent of the Norway-Problem of YAML [0]? Is the space between the key and the type necessary? If not, how to distinguish between objects and types? Does the validation offer some form of unions or mutual exclusion? [0]: https://hitchdev.com/strictyaml/why/implicit-typing-removed/

YAML and its "Arrays" are really broken. The problem I see with Internet Object is that it's also implying this kind of mechanism. Every time I read about new formats, they seem to get either the 1-n relations or the n-n relations implemented well, but not both. I guess that's what's so hard about map/reduce... Regarding YAML: somebody on HN mentioned his project DIXY a couple years ago, and it's much much _much_ eas…

I’ll admit that YAML has its quirks, but a good syntax highlighter can take care of that in my experience. What’s wrong with YAML’s arrays?

Re: Internet Object – A JSON alternative data serialization format

#25

Post by liuliu »

snidane wrote: post #19

Json is a good format to represent results of aggregation queries (group by in sql) using nesting and storing data in a single file. Without that you would need to either 1. store multiple not-nested (tabular, eg. csv) files and join them at the time of use. 2. denormalize all these csvs into a single big csv duplicating the same values over and over. Compression should handle this at storage time, bht you still pay…

You mean, Apache Arrow?

Re: Internet Object – A JSON alternative data serialization format

#26

Post by throwaway81523 »

snidane wrote: post #19

Json is a good format to represent results of aggregation queries (group by in sql) using nesting and storing data in a single file. Without that you would need to either 1. store multiple not-nested (tabular, eg. csv) files and join them at the time of use. 2. denormalize all these csvs into a single big csv duplicating the same values over and over. Compression should handle this at storage time, bht you still pay…

> That said I think adding a trailing comma and comments to json wouldn't be a big stretch.

Sadly, json's designers suffered from the same hubris as the designers of markdown and gemini, when they decided to not include a version number in the file format. So you are kind of hosed if you want to make a change like that.

Before json there was xml (ugh), but before xml there were Lisp S-expressions, which seem to have handled all these issues perfectly well 50 years ago. Yet we keep re-inventing them. Greenspun's tenth law is still with us.

Re: Internet Object – A JSON alternative data serialization format

#29

Post by cookiengineer »

colejohnson66 wrote: post #23
cookiengineer wrote: post #18

Earlier quoted context omitted.

YAML and its "Arrays" are really broken. The problem I see with Internet Object is that it's also implying this kind of mechanism. Every time I read about new formats, they seem to get either the 1-n relations or the n-n relations implemented well, but not both. I guess that's what's so hard about map/reduce... Regarding YAML: somebody on HN mentioned his project DIXY a couple years ago, and it's much much _much_ eas…

I’ll admit that YAML has its quirks, but a good syntax highlighter can take care of that in my experience. What’s wrong with YAML’s arrays?

> What’s wrong with YAML’s arrays?

That there are multiple ways to define Arrays: "- item", "-\n\titem", "\titem" or "item, item" for starters. Parsing YAML into Arrays requires context of its surroundings.

Without the previous context, you cannot know what type of data you're parsing when you are at a "-" at the beginning of a line or a "," in the middle of a line.

This is just unnecessary parser complexity and human ambiguity in my opinion.

As a question to you in case you disagree: What happens when you write down an indented/nested "\t- name: John, Doe"? It's pretty much unpredictable without the previously parsed data structures or their history in YAML.

(I don't wanna start the discussion of "<<" and how it influences the parsing context of YAML data structures. I think the merge key also has no place in a data serialization format.)

Re: Internet Object – A JSON alternative data serialization format

#30

Post by Waterluvian »

We are paying a cost in clarity, human editability, and further splintering of formats.

Everything is a trade off. So what do we get in trade for those rather large costs?

40% bandwidth savings might be worth it. But what are the gzipped comparisons?

Post reply on HN