Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

github.com

Post reply on HN
Print view View original

31–40 of 174 posts

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#31

Post by jonhohle »

tn13 wrote: post #16

This appears to be something in between of JSON and Protocol buffers. I wonder under what conditions Ion makes more sense than either of the JSON/PBuff.

One significant advantage is you can opt-in to sharing schemas - without requiring all consumers to have your schema. Like a lot of Amazon's internal data formats, Ion designed to support backwards compatible schemas as well (that is, adding additional fields does not break existing consumers).

It has isomorphic text and binary representations as part of the standard making debugging or optimized transport a config option.

The type system is significantly richer than JSON and maps well to several languages (internally Amazon uses it with C, C++, Perl, Java, Ruby, etc.).

S-Expressions.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#33

Post by wyc »

jcrites wrote: post #27
wyc wrote: post #23

Earlier quoted context omitted.

I think it depends on what level you're referring to. If you mean record-level, then I concede that it's not self-describing. However, looking at the suggested use cases, it seems that it's "self-describing" in that you'll always be able to decode data stored according to what the documentation recommends: "Avro data is always serialized with its schema. Files that store Avro data should always also include the schem…

That's interesting. I didn't know that about Avro. Does the framework take responsibility for including the schema and defining a format consisting of schema plus data, or is that the responsibility of the application layer? It sounds like that might just be a convention or best practice recommended in the documentation, rather than a technical property of Avro itself. If it's the application's responsibility to bund…

I think libraries generally take care of stuffing the schema into the wire protocol, and I have a hunch you're right in that it's implementation-defined.

I like that in this regard, any individual record in Ion is standalone. I can think of a few ways that could come in handy, e.g., a data packet of nested mixed-version records. Did not know about Paraquet, thanks!

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#34

Post by troyk »

kyllo wrote: post #25
leef wrote: post #11

Finally! I've had to live the JSON nightmare since I left Amazon. Some of the benefits over JSON: * Real date type * Real binary type - no need to base64 encode * Real decimal type - invaluable when working with currency * Annotations - You can tag an Ion field in a map with an annotation that says, e.g. its compression ("csv", "snappy") or its serialized type ('com.example.Foo'). * Text and binary format * Symbol ta…

Real decimal type - invaluable when working with currency What does JavaScript do with this though, just cast it to a float?

move the decimal and use an int (cents in US). It still blows me away that javascript has become so popular on the server without 64bit int types.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#35

Post by jonhohle »

School-Cotton wrote: post #20
kbenson wrote: post #14

Earlier quoted context omitted.

Are we talking about a different YAML, or has Ion existed for ~15 years?

Woah, you're right. I thought YAML was much younger. I stand corrected. Ion is definitely from last decade, at least. So most of the speculation in my post stands.

The JSON RFC was published right around the time Ion was being created making Ion about 10 years old this year. It was not clear at the time that JSON would become so popular ;-) and at the same time Ion fixed a lot of its weak areas (numeric types, dates, struct/object syntax, etc.)

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#37

Post by leef »

kyllo wrote: post #25
leef wrote: post #11

Finally! I've had to live the JSON nightmare since I left Amazon. Some of the benefits over JSON: * Real date type * Real binary type - no need to base64 encode * Real decimal type - invaluable when working with currency * Annotations - You can tag an Ion field in a map with an annotation that says, e.g. its compression ("csv", "snappy") or its serialized type ('com.example.Foo'). * Text and binary format * Symbol ta…

Real decimal type - invaluable when working with currency What does JavaScript do with this though, just cast it to a float?

Ion has functions to turn Ion into JSON which will, of course, lose information. Annotations are dropped, decimals turn into a JSON number type which may lose precision, etc.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#38

Post by tantalor »

voltagex_ wrote: post #28

Open question to anyone reading this: Would you use Ion if you were designing a new house-wide message queue? (e.g. broadcast messages to /Home/Lounge/Lights/ to turn on/off)

No that's overkill, just use JSON.
Post reply on HN