Internet Object – A JSON alternative data serialization format • Hacker News Board

Re: Internet Object – A JSON alternative data serialization format

#51

Post by mccanne » Sun, Oct 24, 2021, 3:11 PM UTC

petre wrote: post #50

mccanne wrote: post #48

Earlier quoted context omitted.

Also, if any of you find problems with the Zed spec(s), we'd love to hear about them. "Now" would be a good time to make changes / fix flaws.

I'd like to see more examples and probably data serialized as zed.

There are a few examples in the ZSON spec...

https://github.com/brimdata/zed/blob/main/docs/formats/zson....

And you can easily see whatever data you'd like formatted as ZSON using the "zq" CLI tool, but I just made this gist (with some data from the brimdata/zed-sample-data report) so you can have a quick look (the bstring stuff is a little noisy and an artifact of the data source being Zeek)... https://gist.github.com/mccanne/94865d557ca3de8abfd3eb09e8ac...

Re: Internet Object – A JSON alternative data serialization format

#52

Post by mofosyne » Sun, Oct 24, 2021, 3:12 PM UTC

Hope it supports semantic tagging like in CBOR

Re: Internet Object – A JSON alternative data serialization format

#53

Post by petre » Sun, Oct 24, 2021, 3:18 PM UTC

wffurr wrote: post #22

random478101 wrote: post #12

As far I can see "IO" addresses the size issue, which is indeed a compression issue for the most part. For a broader take on an alternative, there is concise encoding Concise Encoding [1][2], which I believe addresses a few more issues with existing encodings (clear spec, schema not an afterthought, native support for a variety of data structures, security, ...). [1] https://concise-encoding.org/ [2] The author gave…

People keep saying “just use gzip and JSON is plenty small” but gzip isn’t free. It takes time and power to do all the compression and decompression. The uncompressed size of the data takes up memory on client and server. A smaller data format requires less compression time and power and you can fit more of it in memory at either end.

There's Messagepack and CBOR and Flat Buffers. All of them are faster and smaller than any text based format.

Re: Internet Object – A JSON alternative data serialization format

#54

Post by typingmonkey » Sun, Oct 24, 2021, 3:29 PM UTC

So the plain data is smaller because some information comes from the schema instead of the object. Guess what, you can do the same with json already [1]

[1] https://github.com/pubkey/jsonschema-key-compression

Re: Internet Object – A JSON alternative data serialization format

#55

Post by jdsampayo » Sun, Oct 24, 2021, 3:35 PM UTC

Hurtak wrote: post #41

The whole thing seems to be dead. There is one blog post from 2019 ( https://internetobject.org/the-story/ ) and the Twitter account also was active only in 2019 ( https://twitter.com/InternetObject ).

Moderator should add (2019) to the title, as there has not been any update.

Re: Internet Object – A JSON alternative data serialization format

#56

Post by jFriedensreich » Sun, Oct 24, 2021, 3:51 PM UTC

when a project has more inspirational quotes that tech facts and relation to prior art thats often a red flag.also json is inherently schema less and non binary, this is not a flaw but critical for many usecases. if you want schemas there are many proven alternatives like protobuffs, avro, cap n proto, and message pack.

Re: Internet Object – A JSON alternative data serialization format

#57

Post by kabes » Sun, Oct 24, 2021, 3:54 PM UTC

williamtwild wrote: post #39

kabes wrote: post #34

The example schema has: > age:{int, min:20} Why would a data serialization format bother with data validation like the minimum value here?

front and validation perhaps?

Shouldn't be a concern of a serialization library

Re: Internet Object – A JSON alternative data serialization format

#58

Post by SV_BubbleTime » Sun, Oct 24, 2021, 4:01 PM UTC

kabes wrote: post #57

williamtwild wrote: post #39

Earlier quoted context omitted.

front and validation perhaps?

Shouldn't be a concern of a serialization library

It’s part of the schema part, not the serialization part right? I don’t disagree you parse then validate, but in a schema that defines type of data, it’s not unreasonable to put limits on values.

Re: Internet Object – A JSON alternative data serialization format

#59

Post by 29athrowaway » Sun, Oct 24, 2021, 4:05 PM UTC

cookiengineer wrote: post #18

nikeee wrote: post #13

Since strings don't need to be quoted, what happens during deserialization if you want the string "T"? Does this lead to the equivalent of the Norway-Problem of YAML [0]? Is the space between the key and the type necessary? If not, how to distinguish between objects and types? Does the validation offer some form of unions or mutual exclusion? [0]: https://hitchdev.com/strictyaml/why/implicit-typing-removed/

YAML and its "Arrays" are really broken. The problem I see with Internet Object is that it's also implying this kind of mechanism. Every time I read about new formats, they seem to get either the 1-n relations or the n-n relations implemented well, but not both. I guess that's what's so hard about map/reduce... Regarding YAML: somebody on HN mentioned his project DIXY a couple years ago, and it's much much _much_ eas…

The annoyance of YAML is the possibility of doing things in different ways.

Re: Internet Object – A JSON alternative data serialization format

#60

Post by hiyer » Sun, Oct 24, 2021, 4:10 PM UTC

60% savings won't really count for much when the traffic is compressed, which is the case for most of JSON's uses. For real savings I think you'd have to go with a binary format like protobuf or thrift.

Edit: 50 -> 60