Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Ion Specification

amazon-ion.github.io

Post reply on HN
Print view View original

21–30 of 38 posts

Re: Amazon Ion Specification

#21

Post by jonhohle »

Zenul_Abidin wrote: post #13

Am I right in assuming that this is like Protobuf but just for JSON objects?

It’s a superset of JSON with an isomorphic binary encoding, additional data types (blobs, s-exps, timestamps, symbols, etc.), better number handling, annotations, and the ability to pre-share symbol metadata for more efficient binary encoding (similar to how protobufs encodes fields, but optional).

You can write Ion by hand (like JSON) and share it without a schema (unlike protobufs). There’s fewer ways to express values than YAML, but more data types.

Having S-exps is convenient for writing DSLs in a data language that’s easily readable from other languages.

Re: Amazon Ion Specification

#22

Post by tombert »

postalrat wrote: post #16
nathas wrote: post #15

lol, the internal docs on this at Amazon were something very close to "we invented this before Avro and we think that's probably a better choice if you need binary serialization." My 2 cents: don't use it.

My 2 cents: don't use avro or anything like it unless you can prove its going to save you money

What would you suggest? Just JSON everywhere?

Re: Amazon Ion Specification

#23

Post by throwbadubadu »

steveBK123 wrote: post #17
news_to_me wrote: post #12

Saw this the other day, but the multiple types of null kind of turned me off - e.g. `null.int`, `null.float`, `null.null`. Is there a good justification for this? Seems like a kluge in any case.

typed nulls good

Sounds like the grug brained developer speaking... Hi! ;)

Re: Amazon Ion Specification

#24

Post by postalrat »

tombert wrote: post #22
postalrat wrote: post #16

Earlier quoted context omitted.

My 2 cents: don't use avro or anything like it unless you can prove its going to save you money

What would you suggest? Just JSON everywhere?

json, csv, text, html, binary blobs you dont create, whatever is easiest

Re: Amazon Ion Specification

#25

Post by rahkiin »

tombert wrote: post #22
postalrat wrote: post #16

Earlier quoted context omitted.

My 2 cents: don't use avro or anything like it unless you can prove its going to save you money

What would you suggest? Just JSON everywhere?

I’m interested in the answer as well. Also interested what’s wrong with Ion

Re: Amazon Ion Specification

#26

Post by spaceywilly »

news_to_me wrote: post #12

Saw this the other day, but the multiple types of null kind of turned me off - e.g. `null.int`, `null.float`, `null.null`. Is there a good justification for this? Seems like a kluge in any case.

Seems like the justification would be to keep the type information when going back and forth to Ion. More like "multiple nullable types" instead of "multiple types of null"

userBirthDay: null userBirthDay: null.timestamp <-- ok, it's a timestamp typed variable, but we don't know the value. Yay, happy programmer.

Re: Amazon Ion Specification

#27

Post by tombert »

postalrat wrote: post #24
tombert wrote: post #22

Earlier quoted context omitted.

What would you suggest? Just JSON everywhere?

json, csv, text, html, binary blobs you dont create, whatever is easiest

I have never used Ion so I cannot speak to its use in practice, but I haven't really had too much of an issue with msgpack. It's faster than JSON, more compressed than JSON, without being any more difficult than any JSON library I've used. It's an almost-universal good for me; the only thing you lose is the ability to easily introspect the messages if there's an issue.

Re: Amazon Ion Specification

#28

Post by Waterluvian »

postalrat wrote: post #24
tombert wrote: post #22

Earlier quoted context omitted.

What would you suggest? Just JSON everywhere?

json, csv, text, html, binary blobs you dont create, whatever is easiest

cbor even.

Honestly, if you’re in a case where you absolutely know none of these work for you and you can absolutely prove you need another, you’re probably just going to write your own. And that’s a fleetingly rare case.

Re: Amazon Ion Specification

#29

Post by jesterpm »

tombert wrote: post #20

I would be interested to see how this compares to something like msgpack [1] in performance and final size of the binary. Msgpack has been my go-to for binary serialization for years due to how simple and fast it is, and how easy it is to make it work with native Clojure data structures. [1] https://msgpack.org/index.html

That comparison would depend heavily on what you're storing.

Ion has the option of using symbol tables to replace strings (e.g. in struct/map keys or in values). So, if you benchmark had a large number of records with similar structures, I would expect Ion to pull ahead. On the other hand, if each record had nothing in common, I'd expect them to perform similarly.

One feature of the Ion libraries that I've liked is the parser will take any of the formats and figure out what to do with it (text, binary, compressed binary). It's one less thing to worry about. You can switch encodings later without breaking consumers, you can write plain text Ion when you're testing, etc.

Post reply on HN