Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

github.com

Post reply on HN
Print view View original

131–140 of 174 posts

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#131

Post by cm3 »

voltagex_ wrote: post #29
cm3 wrote: post #8

A question for frontend devs: Will H2 being binary on the wire inspire more use of binary data representations as well, with conversion to JSON only on the client? Passing around JSON or XML across a big SOA (or micro-services) architecture is a waste of cycles and doesn't have types attached for reliability and security.

Do you mean passing around binary between backend services and then having a binary->JSON "proxy" behind whatever is receiving AJAX requests from the client?

My idea was that the client (HTML+JS) will transform the binary data into JSON or skip the conversion and process it directly. Seeing how fast JS engines have become and the amount of typed binary arrays processed in JavaScript, I believe it's a viable approach. But I'm not a frontend dev, so I can't be certain.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#132

Post by oblio »

buro9 wrote: post #123
oblio wrote: post #113

Earlier quoted context omitted.

What's the scale in this context? Your explanation doesn't really clarify.

From two different comments on here: http://stackoverflow.com/questions/5689369/what-is-the-diffe... Precision is the number of significant digits. Oracle guarantees the portability of numbers with precision ranging from 1 to 38. Scale is the number of digits to the right (positive) or left (negative) of the decimal point. The scale can range from -84 to 127. Worth noting this isn't specifically an Oracle thing, most…

Cool, I didn't know about this technique. Thanks for the detailed explanation!

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#133

Post by anentropic »

buro9 wrote: post #101
kyllo wrote: post #25

Earlier quoted context omitted.

Real decimal type - invaluable when working with currency What does JavaScript do with this though, just cast it to a float?

The real way is: "price": { "amount": "1500", "scale": 2, "symbol": "GBP", } Currency has 3 properties, the amount, scale, and symbol. Amount is a string, it holds a bigint. Yes, it's a string. The value of Scale can be up to 5 but is usually 2 or 3. Symbol is the ISO code. Whenever I see a financial system that uses "amount": 15.00 I know that the system is ill-conceived.

what is the point of sending integer "amount" as a string?

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#134

Post by aeroevan »

jcrites wrote: post #27
wyc wrote: post #23

Earlier quoted context omitted.

I think it depends on what level you're referring to. If you mean record-level, then I concede that it's not self-describing. However, looking at the suggested use cases, it seems that it's "self-describing" in that you'll always be able to decode data stored according to what the documentation recommends: "Avro data is always serialized with its schema. Files that store Avro data should always also include the schem…

That's interesting. I didn't know that about Avro. Does the framework take responsibility for including the schema and defining a format consisting of schema plus data, or is that the responsibility of the application layer? It sounds like that might just be a convention or best practice recommended in the documentation, rather than a technical property of Avro itself. If it's the application's responsibility to bund…

Avro files have a header which has metadata including the schema as well as things like compression codec (supports deflate and snappy) and all of the implementations that I have used (java and python bindings mostly) just does this in the background.

Another fun thing is that avro supports union types, so to make things nullable you just union[null, double] or whatever.

But one of the best things about avro (and parquet for that matter) is that it is well supported by the hadoop ecosystem

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#135

Post by Matt3o12_ »

anentropic wrote: post #133
buro9 wrote: post #101

Earlier quoted context omitted.

The real way is: "price": { "amount": "1500", "scale": 2, "symbol": "GBP", } Currency has 3 properties, the amount, scale, and symbol. Amount is a string, it holds a bigint. Yes, it's a string. The value of Scale can be up to 5 but is usually 2 or 3. Symbol is the ISO code. Whenever I see a financial system that uses "amount": 15.00 I know that the system is ill-conceived.

what is the point of sending integer "amount" as a string?

So that it can be an integer with an arbitrary length, or a float/double without precision problems. You can also let our own integer classes do the parsing (which might even be able tonhandle complex types).

After all, everything in JSON is a string since it doesn't have a binary format and it shouldn't cause a huge overhead to do the parsing yourself (that might depend on the library, though).

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#136

Post by bct »

pjmlp wrote: post #106
JonathonW wrote: post #89

Earlier quoted context omitted.

Plists are nifty, but the text format's XML-based, which makes it too complex and too verbose to be a general-purpose alternative to something like JSON. (plutil "supports" a json format, but it's not capable of expressing the complete feature set of the XML or binary formats.)

I don't get this gripe with XML, it is meant to be used by tools not to be written by hand. Where is the XPath and XQuery for JSON? Do people really think that manually iterating over the whole JSON document to find the data or writing yet another parser, is better?

The XML serialization Apple defined for plists is awful and not easy to query with XPath.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#137

Post by sswezey »

School-Cotton wrote: post #111
nikolay wrote: post #50

Earlier quoted context omitted.

Okay, but they did a really poor job marketing it in this release. Plus, if it's used within Amazon, why it's Java-only so far?

I'm sure there are ion bindings for every language in common use at Amazon. But a huge percentage of Amazon code is Java, so presumably this one was the best maintained and documented.

I doubt it, when I was there Ion was only used by only a handful of Java teams doing backend work. It was also horribly documented and supported at the time (3.5 years ago).

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#138

Post by eyan »

LVB wrote: post #77
eyan wrote: post #74

Surprised nobody mentioned CBOR ( http://cbor.io ) yet. Aka RFC 7049 ( http://tools.ietf.org/html/rfc7049 ).

It is referenced in the Ion docs: http://amznlabs.github.io/ion-docs/index.html

@LVB, thanks for that. RTFM-ing made me think twice about adopting CBOR or going with Ion. I'll also mention Velocypack (https://github.com/arangodb/velocypack) while here.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#139

Post by mjt0229 »

jonhohle wrote: post #42

Big congrats to Todd, Almann, Chris, Henry, and everyone else who made this happen. Several years ago, I wouldn't have imagined this possible and I'm a little bummed that I left before it happened. Like leef said above, I'm glad to have Ion as an option again.

I was there at the beginning of Ion back in my Amazon days. We may well use it in my current job, too!

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#140

Post by jasonpeacock »

kazinator wrote: post #84

I Consider this Harmful (TM) and will oppose the adoption in every organization where I have an opportunity to voice such. (In its present form, to be clear!) There is no need to have a null which is fragmented into null.timestamp, null.string and whatever. It will complicate processing. Just because you know the type of some element is timestamp, you must worry whether or not it is null and what that means. There sh…

Actually, ION was originally developed for C++ and took quite a while to gain Java support.

Source: I was there.

Post reply on HN