Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

github.com

Post reply on HN
Print view View original

51–60 of 174 posts

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#51

Post by duskwuff »

vparikh wrote: post #30

Wasn't this solved already by the BSON specification - http://bsonspec.org ? Sure this allows you a definition of types, but this could easily be done using standard JSON meta data for each field. I find BSON simpler and more elegant.

BSON is awful.

* It doesn't have "true" types in the sense that Ion does. It's basically just a binary serialization of JSON, with extra stuff.

* Despite being a binary format, it's actually bulkier than JSON in most situations.

* It removes any semblance of canonicity from many representations. A number, for instance, can potentially be represented by any of at least 3 types (double, int32, and int64).

* It has signed 32-bit length limits all over the place. Not that I'd want to be storing 2GB of data in a single JSON document either, but it's not even possible to do so with BSON!

* It requires redundant null bytes in unpredictable places. For instance, all strings must be stored with a trailing null byte, which is included in their length. There's also a trailing null byte at the end of a document for no reason at all.

* It is unabashedly Javascript-specific, containing types like "JavaScript code with scope" which are meaningless to other languages.

* It also contains some MongoDB-specific cruft, such as the "ObjectID" and "timestamp" types (the latter of which, despite its name, cannot actually be used to store time values).

* It contains numerous "deprecated" and "old" features (in version 1.0!) with no guidance as to how implementations should handle them.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#52

Post by hartror »

leef wrote: post #11

Finally! I've had to live the JSON nightmare since I left Amazon. Some of the benefits over JSON: * Real date type * Real binary type - no need to base64 encode * Real decimal type - invaluable when working with currency * Annotations - You can tag an Ion field in a map with an annotation that says, e.g. its compression ("csv", "snappy") or its serialized type ('com.example.Foo'). * Text and binary format * Symbol ta…

Does it support comments?

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#53

Post by leef »

hartror wrote: post #52
leef wrote: post #11

Finally! I've had to live the JSON nightmare since I left Amazon. Some of the benefits over JSON: * Real date type * Real binary type - no need to base64 encode * Real decimal type - invaluable when working with currency * Annotations - You can tag an Ion field in a map with an annotation that says, e.g. its compression ("csv", "snappy") or its serialized type ('com.example.Foo'). * Text and binary format * Symbol ta…

Does it support comments?

Yes - http://amznlabs.github.io/ion-docs/spec.html

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#55

Post by makoz »

nikolay wrote: post #50
leef wrote: post #11

Finally! I've had to live the JSON nightmare since I left Amazon. Some of the benefits over JSON: * Real date type * Real binary type - no need to base64 encode * Real decimal type - invaluable when working with currency * Annotations - You can tag an Ion field in a map with an annotation that says, e.g. its compression ("csv", "snappy") or its serialized type ('com.example.Foo'). * Text and binary format * Symbol ta…

Okay, but they did a really poor job marketing it in this release. Plus, if it's used within Amazon, why it's Java-only so far?

Amazon's mainly a Java shop, not sure if that helps you.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#56

Post by machinagod »

jonhohle wrote: post #42

Big congrats to Todd, Almann, Chris, Henry, and everyone else who made this happen. Several years ago, I wouldn't have imagined this possible and I'm a little bummed that I left before it happened. Like leef said above, I'm glad to have Ion as an option again.

Hear, hear! Posted it as I've been raving about ion (particularly s-exp support) to non-amazonians, but credit goes all to them.

It's particularly interesting to see the fixes and improvements from the actual open source cleanup effort getting to (many) Internal production services.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#57

Post by tjonker »

Ericson2314 wrote: post #9

> Decimal maintains precision: -0. != -0.0 What? This means their "arbitrary-precision decimals" are actually isomorphic to (Rational x Natural).

The "!=" means "not the same value according to the Ion data model".

The Ion value 0.0 has one digit of precision (after the decimal point), while the value 0.00 has two. In the Ion data model, those are two distinct values, and conforming implementations must maintain the distinction.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#58

Post by jonhohle »

blake8086 wrote: post #39

How does Ion help with schema evolution? I see it mentioned, but not described.

In practice there are three properties that help with schema evolution:

    1) open types - typically applications consuming Ion data 
       do  not restrict the fields included (that is, they 
       gracefully ignore, and often even pass along additional 
       fields). Schemas may grow while being backwards 
       compatible with existing software.
    2) Type annotations allow embedding schema information into 
       a datagram without the need for agreeing on special 
       fields. Datagrams may have multiple values at the top 
       level, so its possible to provide multiple 
       representations without introducing a new top-level 
       container.
    3) The only data might need to be shared between a producer 
       and consumer is a SymbolTable which may be applicable to 
       several schemas and may be shared inline if necessary.
       Otherwise, objects in a datagram are always inspectable 
       and discoverable without additional metadata.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#59

Post by viraptor »

So far, most of the interesting bits I see in Ion are covered in YAML (which is also JSON-superset). Most of the rest are extra types, which YAML allows you to implement. The only really missing bit is the binary encoding... but that seems unrelated to the text format itself.

This really looks like a NIH specification.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#60

Post by haberman »

To think about the difference between serialization formats, here's an analogy I hope will help.

Protocol Buffers (and I think Thrift, and maybe Avro) are sort of like C or C++: you declare your types ahead of time, and then you take some binary payload and "cast" it (parse it actually) into your predefined type. If those bytes weren't actually serialized as that type, you'll get garbage. On the plus side, the fact that you declared your types statically means that you get lots of useful compile-time checking and everything is really efficient. It's also nice because you can use the schema file (ie. .proto files) to declare your schema formally and document everything.

JSON and Ion are more like a Python/Javascript object/dict. Objects are just attribute-value bags. If you say it has field fooBar at runtime, now it does! When you parse, you don't have to know what message type you are expecting, because the key names are all encoded on the wire. On the downside, if you misspell a key name, nothing is going to warn you about it. And things aren't quite as efficient because the general representation has to be a hash map where every value is dynamically typed. On the plus side, you never have to worry about losing your schema file.

I think this is a case where "strongly typed" isn't the clearest way to think about it. It's "statically typed" vs. "dynamically typed" that is the useful distinction.

Post reply on HN