Live data from Hacker News

Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

github.com

121–130 of 174 posts

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#121
post #55

Earlier quoted context omitted.

Amazon's mainly a Java shop, not sure if that helps you.

Not really the case - a lot of major projects (like boto and AWS CLI) are in Python.

These are only client-side interfaces. The server-side is usually much larger.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#122

Open question to anyone reading this: Would you use Ion if you were designing a new house-wide message queue? (e.g. broadcast messages to /Home/Lounge/Lights/ to turn on/off)

Maybe, when Ion gets supports for most major languages. I won't touch it now because it means to go Java for every application that reads or writes to that queue. Not because of Java, but because it's only one language. It should get on par with support for the other formats listed in the comments before one should be confident to use it.

At least the JVM supports multiple languages (Scala, Clojure etc.), but if there's a spec it shouldn't be hard for anyone to add support for other languages.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#123
post #113
post #101

Earlier quoted context omitted.

The real way is: "price": { "amount": "1500", "scale": 2, "symbol": "GBP", } Currency has 3 properties, the amount, scale, and symbol. Amount is a string, it holds a bigint. Yes, it's a string. The value of Scale can be up to 5 but is usually 2 or 3. Symbol is the ISO code. Whenever I see a financial system that uses "amount": 15.00 I know that the system is ill-conceived.

What's the scale in this context? Your explanation doesn't really clarify.

From two different comments on here: http://stackoverflow.com/questions/5689369/what-is-the-diffe...

   Precision is the number of significant digits. Oracle guarantees the
   portability of numbers with precision ranging from 1 to 38.

   Scale is the number of digits to the right (positive) or left (negative)
   of the decimal point. The scale can range from -84 to 127.
Worth noting this isn't specifically an Oracle thing, most financial systems need to be sure that it can store currency numbers accurately and this convention is widely used to ensure this.

And:

   Precision 4, scale 2: 99.99

   Precision 10, scale 0: 9999999999

   Precision 8, scale 3: 99999.999

   Precision 5, scale -3: 99999000
Typically when dealing with currencies scale is only used to represent the units less than whole unit of the currency, i.e. cents and pence. But there isn't anything that restricts it from being used to accommodate larger numbers with the use of negative scales.

A current list of all ISO 4217 codes and the currency properties can be found here http://www.currency-iso.org/en/home/tables/table-a1.html

i.e.

   
      UNITED STATES OF AMERICA (THE)
      US Dollar
      USD
      840
      2
   
The CcyMnrUnts property denotes 2 decimal places for the sub-unit of the dollar.

So for the above example of 99999.999 you would store an amount of 99999999 and a scale of 3.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#124

Earlier quoted context omitted.

Not really the case - a lot of major projects (like boto and AWS CLI) are in Python.

These are only client-side interfaces. The server-side is usually much larger.

True, but my point was that there's enough talent at Amazon, working on SDKs, and others, and there are precedents where even more complex projects such as JMESPath have wide support [0].

[0]: http://jmespath.org/libraries.html

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#125

Earlier quoted context omitted.

That's a good description, but I'd say that we have a strongly weakly typed axis and a statically dynamically typed axis here. Or I might actually prefer to name the first axis poorly richly typed. poorly typed richly typed dynamic CSV, INI JSON YAML, Ion static Bencode, ASN.1 Protobuf What I mean by "richly typed" is that you would never read a timestamp off the wire and not know that it's a timestamp. By comparison…

Sure, I think your "poorly typed" vs. "richly typed" axis just refers to how many built-in types it has. It's true that CSV and INI only have one type (string). And it's true that when more types are built in, you have fewer cases where you have to just stuff your data into a specially-formatted string.

Yes, that's exactly what I said.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#127
post #87

Earlier quoted context omitted.

> By your definition, any language with strings is weakly typed, since you can always interpret a string as being something else No, I wouldn't say that's the case. For example, in PHP you can literally write: if (1 == "1") { ... ... and the condition evaluates to true. You can do similar things in Excel; Excel doesn't even really differentiate between those two values in the first place. (At least that's how it seem…

> You can't misuse any kind of Ion value that is a string as if it were a timestamp without performing an explicit conversion. The same is true of JSON. There is no difference, except that Ion has a timestamp type and JSON does not. If you disagree, please identify what characteristic of Ion's design makes it more strongly typed than JSON, other than the set of types that is built in.

You are choosing a definition of strong typing that supports your argument, but the argument is over the meaning of strong typing to begin with. It's not as if there's some universally accepted definition of strong typing. Like functional programming, functional purity, object oriented, etc.—none of these terms are universally defined.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#128
post #27
post #23

Earlier quoted context omitted.

I think it depends on what level you're referring to. If you mean record-level, then I concede that it's not self-describing. However, looking at the suggested use cases, it seems that it's "self-describing" in that you'll always be able to decode data stored according to what the documentation recommends: "Avro data is always serialized with its schema. Files that store Avro data should always also include the schem…

That's interesting. I didn't know that about Avro. Does the framework take responsibility for including the schema and defining a format consisting of schema plus data, or is that the responsibility of the application layer? It sounds like that might just be a convention or best practice recommended in the documentation, rather than a technical property of Avro itself. If it's the application's responsibility to bund…

In the spec[1] there is a definition of an "object container file" which includes the schema, and is the default format used whenever you save an Avro file. You can even use it whenever sending Avro data through the wire, if you don't mind paying the extra space cost.

[1]: http://avro.apache.org/docs/1.7.7/spec.html

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#129

I Consider this Harmful (TM) and will oppose the adoption in every organization where I have an opportunity to voice such. (In its present form, to be clear!) There is no need to have a null which is fragmented into null.timestamp, null.string and whatever. It will complicate processing. Just because you know the type of some element is timestamp, you must worry whether or not it is null and what that means. There sh…

> It will complicate processing. In practice, it doesn't. If you want to know if an IonValue is null, ask it with #isNull. If you don't care about the null's type, ignore it. On the other hand, the type is an additional form of metadata which allows overloading the meaning of a value. nulls can also be annotated, so Ion doesn't really have the concept of a singular shared null sentinel. More so than JSON, Ion often u…

I think the concern is that if you take an IonValue and cast it to IonText, then call stringValue(), you'll get an exception somewhere if the document contained a null value.

It recalls the nullability arguments between the ML family and the C/Java family.

kazinator is asking for safer document semantics and a type-safe API.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#130
post #11

Finally! I've had to live the JSON nightmare since I left Amazon. Some of the benefits over JSON: * Real date type * Real binary type - no need to base64 encode * Real decimal type - invaluable when working with currency * Annotations - You can tag an Ion field in a map with an annotation that says, e.g. its compression ("csv", "snappy") or its serialized type ('com.example.Foo'). * Text and binary format * Symbol ta…

You could have used CBOR for many of those things (http://cbor.io/).
Post reply on HN