Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Ion Specification

amazon-ion.github.io

Post reply on HN
Print view View original

11–20 of 38 posts

Re: Amazon Ion Specification

#11

Post by abeppu »

jauntywundrkind wrote: post #4
mikece wrote: post #2

Am I reading this right that it's a binary format for real-time streaming data, similar to Avro, but can include arbitrarily deep nested structures unlike Avro?

Why do you single out avro & not any of the hundreds of other ser/de systems? Is that what you know best? Is there something specific about avro that makes it feel particularly similar? https://github.com/maximveksler/awesome-serialization

In fairness, in the list you link to,

- Avro and Ion are the only two that are labeled Textual/Binary

- They are in the same Big Data grouping

- They both are schema-embedded, and support some rich nested datastructures, though they deviate on many of the specifics

So I think it's reasonable to pick out Avro as an especially similar point of comparison.

Re: Amazon Ion Specification

#15

Post by nathas »

lol, the internal docs on this at Amazon were something very close to "we invented this before Avro and we think that's probably a better choice if you need binary serialization."

My 2 cents: don't use it.

Re: Amazon Ion Specification

#16

Post by postalrat »

nathas wrote: post #15

lol, the internal docs on this at Amazon were something very close to "we invented this before Avro and we think that's probably a better choice if you need binary serialization." My 2 cents: don't use it.

My 2 cents: don't use avro or anything like it unless you can prove its going to save you money

Re: Amazon Ion Specification

#18

Post by kevan »

satvikpendem wrote: post #14

Can this be used similarly to GraphQL?

This is just the data serialization format, you have to build any other functionality yourself. We do have a pattern on a few of our APIs where there's a big fixed schema (i.e. it's just a struct and you can't do GraphQL things like following references and hydrating them into objects) and clients select the subset of attributes they want and we only return that. It's useful for reducing response sizes but the main benefit is we can pretty easily track which attributes are actually used over time. That helps us deprecate attributes with a lot less pain.

Re: Amazon Ion Specification

#19

Post by zyang »

news_to_me wrote: post #12

Saw this the other day, but the multiple types of null kind of turned me off - e.g. `null.int`, `null.float`, `null.null`. Is there a good justification for this? Seems like a kluge in any case.

My guess is buffer size calculations.

Re: Amazon Ion Specification

#20

Post by tombert »

I would be interested to see how this compares to something like msgpack [1] in performance and final size of the binary. Msgpack has been my go-to for binary serialization for years due to how simple and fast it is, and how easy it is to make it work with native Clojure data structures.

[1] https://msgpack.org/index.html

Post reply on HN