Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Ion – A richly-typed, self-describing, hierarchical serialization format

amzn.github.io

Post reply on HN
Print view View original

111–120 of 240 posts

Re: Amazon Ion – A richly-typed, self-describing, hierarchical serialization format

#111

Post by sirk390 »

otabdeveloper4 wrote: post #102

Nice! This thing is actually sane and thought through. A first for serialization formats. They're usually a shitshow. (Should have gone with 'rational' instead of 'decimal', though. Decimal will be too painful to implement accross languages and implementations. Java bias?)

But decimal are way more useful as they can represent currency amounts. It would be strange to show a currency amount like "3/4" or "11/12". Personally, the two datatypes I have always been adding manually to json are datetimes and decimals (from python)

Re: Amazon Ion – A richly-typed, self-describing, hierarchical serialization format

#112

Post by valenterry »

joshlemer wrote: post #110
valenterry wrote: post #84

Earlier quoted context omitted.

> There's no static type layer over top of this Yeah, that's the problem. I mean, hey, why json? We could just use unstructured plaintext for everything and now we are free to do everything. But obviously that has its own drawbacks. Having built-in support for sumtypes means better and more ergonomic support from libraries, it means there is one standard and not different ways to encode things and it also means bette…

The point is that there's no reason to single out sumtypes here. Insofar as ions/json has support for arrays/objects/strings/numbers, it has exactly the same support for sumtypes, as in the example I showed above. Here is a list of "sumtype" `string | number | object`: [{}, "hi", 1, 2, 3, "yo", {a: "bc"}]

No, that is not a sumtype, that's an array.

In the same sense "1e-12" is not a number, it's a string. Yes, it's a string that encodes a number in a certain notion, but for alle the tooling, the IDE, the libraries, etc. it will stay a string.

Re: Amazon Ion – A richly-typed, self-describing, hierarchical serialization format

#113

Post by joshlemer »

valenterry wrote: post #112
joshlemer wrote: post #110

Earlier quoted context omitted.

The point is that there's no reason to single out sumtypes here. Insofar as ions/json has support for arrays/objects/strings/numbers, it has exactly the same support for sumtypes, as in the example I showed above. Here is a list of "sumtype" `string | number | object`: [{}, "hi", 1, 2, 3, "yo", {a: "bc"}]

No, that is not a sumtype, that's an array. In the same sense "1e-12" is not a number, it's a string. Yes, it's a string that encodes a number in a certain notion, but for alle the tooling, the IDE, the libraries, etc. it will stay a string.

What I mean is, it is an array of a sumtype `number | string | object`. So precisely, you could call it a `list`

Re: Amazon Ion – A richly-typed, self-describing, hierarchical serialization format

#114

Post by valenterry »

the_girabbit wrote: post #46
clhodapp wrote: post #38

It's staggering to me that people keep making these "rich" data formats without sum types. At least to me, the "ors" are just as important as the "ands" in domain modeling. Apart from that, while you can always sort of fake it with a bunch of optional fields I believe that you kind of need a native encoding to a tagged union if you want to avoid bloating your messages.

Genuine question—why would you need a sum type in a self-describing data format?

Well, there are already sumtypes, just only specific builtin ones, not custom ones. E.g. booleans are sumtypes (true | false). Everything else that is nullable is also a sumtype (e.g. number | null).

I think it should be pretty obvious how these are helpful and why they are needed no?

Re: Amazon Ion – A richly-typed, self-describing, hierarchical serialization format

#115

Post by adwn »

tomerv wrote: post #109
logicchains wrote: post #107

Earlier quoted context omitted.

>It'll have unintentional collisions if you ever generate more than 4 billion of these random numbers. If it's 64 bit, doesn't that mean you'd need to generate ~10000000000000000000000000000000000000000000000000000000000000000 (2^64) of those numbers to have a collision, not 2^32?

If you generate randomly then, due to the birthday paradox, after generating sqrt(N) values you have a reasonable chance of collision. The birthday paradox is named after the non-intuitive fact that with just 32 people in a room you have > 50% of 2 people having a birthday on the same day of the year.

> 32 people

Slight correction: only 23 people, actually. So in every second football ("soccer") game, you have two people on the field with the same birthday.

Re: Amazon Ion – A richly-typed, self-describing, hierarchical serialization format

#116

Post by quantumspandex »

joshlemer wrote: post #110
valenterry wrote: post #84

Earlier quoted context omitted.

> There's no static type layer over top of this Yeah, that's the problem. I mean, hey, why json? We could just use unstructured plaintext for everything and now we are free to do everything. But obviously that has its own drawbacks. Having built-in support for sumtypes means better and more ergonomic support from libraries, it means there is one standard and not different ways to encode things and it also means bette…

The point is that there's no reason to single out sumtypes here. Insofar as ions/json has support for arrays/objects/strings/numbers, it has exactly the same support for sumtypes, as in the example I showed above. Here is a list of "sumtype" `string | number | object`: [{}, "hi", 1, 2, 3, "yo", {a: "bc"}]

His point was type support and standard way of doing things. Using your argument we just need string type to represent everything.

Re: Amazon Ion – A richly-typed, self-describing, hierarchical serialization format

#117

Post by spenczar5 »

clhodapp wrote: post #108
spenczar5 wrote: post #105

Earlier quoted context omitted.

Others have mentioned Protobuf and Capnproto's support. Avro has them too, they're called Union. It seems that sum types are the norm, actually.

Those do now but I believe that all of them added support years after their initial versions

I think you're incorrect:

Avro had unions in version 1.0 [0], which is from 2012.

Capnproto had unions back in 2013 [1]. That's from the v0.1 days, or maybe even earlier.

Protobuf has had oneof support for about 7 years. They were added in version 2.6.0, from 2014-08-15 [2]. That's still 6 years after the initial public release in 2008, though, so this is maybe what you were thinking of? I don't know too many people who were using protobuf in those days outside of Google, though.

---

[0] https://avro.apache.org/docs/1.0.0/spec.html#Unions

[1] https://github.com/capnproto/capnproto/commit/eb8404a157e074...

[2] https://github.com/protocolbuffers/protobuf/blob/master/CHAN...

Re: Amazon Ion – A richly-typed, self-describing, hierarchical serialization format

#118

Post by doo_daa »

tomerv wrote: post #109
logicchains wrote: post #107

Earlier quoted context omitted.

>It'll have unintentional collisions if you ever generate more than 4 billion of these random numbers. If it's 64 bit, doesn't that mean you'd need to generate ~10000000000000000000000000000000000000000000000000000000000000000 (2^64) of those numbers to have a collision, not 2^32?

If you generate randomly then, due to the birthday paradox, after generating sqrt(N) values you have a reasonable chance of collision. The birthday paradox is named after the non-intuitive fact that with just 32 people in a room you have > 50% of 2 people having a birthday on the same day of the year.

I think it's 23 people in a room. The canonical example is people on a football (soccer) pitch. With 11 per side plus the referee there's a 50% chance that two will share the same birthday.
Post reply on HN