Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

github.com

Post reply on HN
Print view View original

81–90 of 174 posts

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#81

Post by haberman »

tjonker wrote: post #78
haberman wrote: post #73

Earlier quoted context omitted.

Ion has more data types than JSON, it's true. Ion has a timestamp type and JSON does not, so you could say it's "richer" if you want, but that just means "it has more types." However I don't think it's accurate to say that the typing of Ion is any "stronger." Both Ion and JSON are fully dynamically typed, which means that types are attached to every value on the wire. It's just that without an actual timestamp type i…

I generally agree, except the "type" of JSON numbers isn't well-defined with respect to precision and binary-vs-decimal floating point representation. An application that cares deeply about either aspect of numbers can't rely on JSON alone to ensure that the values are properly interpreted by all consumers.

That is a good point in that it is a very accurate reading of the JSON spec. In practice many (even most) JSON implementations don't give applications access to any precision beyond what an IEEE double can represent. So while you may take advantage of arbitrary precision in JSON and be fine according to the spec, your users will probably suffer data loss unless they are very picky about what JSON library they use. For example, JSON.parse() in JavaScript is out.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#82

Post by userbinator »

tantalor wrote: post #38
voltagex_ wrote: post #28

Open question to anyone reading this: Would you use Ion if you were designing a new house-wide message queue? (e.g. broadcast messages to /Home/Lounge/Lights/ to turn on/off)

No that's overkill, just use JSON.

Even JSON is overkill for an application like that. Pure binary would be my choice.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#83

Post by haberman »

jcrites wrote: post #76
haberman wrote: post #73

Earlier quoted context omitted.

Ion has more data types than JSON, it's true. Ion has a timestamp type and JSON does not, so you could say it's "richer" if you want, but that just means "it has more types." However I don't think it's accurate to say that the typing of Ion is any "stronger." Both Ion and JSON are fully dynamically typed, which means that types are attached to every value on the wire. It's just that without an actual timestamp type i…

The notions of "strong" and "weak" typing have never been particularly well-defined, but I think my usage is in line with their usual meaning: https://en.wikipedia.org/wiki/Strong_and_weak_typing > Some programming languages make it easy to use a value of one type as if it were a value of another type. This is sometimes described as "weak typing". Strong typing makes it difficult to use a value of one type as if it w…

By your definition, any language with strings is weakly typed, since you can always interpret a string as being something else. Strongly/weakly typed has never been a particularly useful description (as the page you linked notes), and I think it's particularly unhelpful here.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#84

Post by kazinator »

I Consider this Harmful (TM) and will oppose the adoption in every organization where I have an opportunity to voice such. (In its present form, to be clear!)

There is no need to have a null which is fragmented into null.timestamp, null.string and whatever. It will complicate processing. Just because you know the type of some element is timestamp, you must worry whether or not it is null and what that means.

There should be just one null value, which is its own type. A given datum is either permitted to be null OR something else like a string. Or it isn't; it is expected to be a string, which is distinct from the null value; no string is a null value.

It's good to have a read notation for a timestamp, but it's not an elementary type; a timestamp is clearly an aggregate and should be understood as corresponding to some structure type. A timestamp should be expressible using that structure, not only as a special token.

This monstrosity is not exhibiting good typing; it is not good static typing, and not good dynamic typing either. Under static typing we can have some "maybe" type instead of null.string: in some representations we definitely have a string. In some other places we have a "maybe string", a derived type which gives us the possibility that a string is there, or isn't. Under dynamic typing, we can superimpose objects of different type in the same places; we don't need a null version of string since we can have "the" one and only null object there.

This looks like it was invented by people who live and breathe Java and do not know any other way of structuring data. Java uses statically typed references to dynamic objects, and each such reference type has a null in its domain so that "object not there" can be represented. But just because you're working on a reference implementation in such a language doesn't mean you cannot transcend the semantics of the implementation language. If you want to propose some broad interoperability standard, you practically must.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#85

Post by tjonker »

viraptor wrote: post #59

So far, most of the interesting bits I see in Ion are covered in YAML (which is also JSON-superset). Most of the rest are extra types, which YAML allows you to implement. The only really missing bit is the binary encoding... but that seems unrelated to the text format itself. This really looks like a NIH specification.

Ion's equivalently-expressive text and binary formats is absolutely central to its design, and IMO one of its most compelling features. You don't have to choose between "human readable" or "compact and fast", you can switch between them at will. This helps Ion meet the requirements of a broader set of applications, eliminating the cost and complexity and impedance-mismatch problems you get by transforming between multiple serialization formats.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#86

Post by justin_oaks »

troyk wrote: post #34
kyllo wrote: post #25

Earlier quoted context omitted.

Real decimal type - invaluable when working with currency What does JavaScript do with this though, just cast it to a float?

move the decimal and use an int (cents in US). It still blows me away that javascript has become so popular on the server without 64bit int types.

Do you really need those extra 11 bits? Javascript numbers accurately represent integers up to 2^53 - 1. See https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#87

Post by jcrites »

haberman wrote: post #83
jcrites wrote: post #76

Earlier quoted context omitted.

The notions of "strong" and "weak" typing have never been particularly well-defined, but I think my usage is in line with their usual meaning: https://en.wikipedia.org/wiki/Strong_and_weak_typing > Some programming languages make it easy to use a value of one type as if it were a value of another type. This is sometimes described as "weak typing". Strong typing makes it difficult to use a value of one type as if it w…

By your definition, any language with strings is weakly typed, since you can always interpret a string as being something else. Strongly/weakly typed has never been a particularly useful description (as the page you linked notes), and I think it's particularly unhelpful here.

> By your definition, any language with strings is weakly typed, since you can always interpret a string as being something else

No, I wouldn't say that's the case. For example, in PHP you can literally write:

  if (1 == "1") { ...
... and the condition evaluates to true. You can do similar things in Excel; Excel doesn't even really differentiate between those two values in the first place. (At least that's how it seems as a casual user.)

This is not the case in strongly typed programming languages that have strings such as C++ or Java. You can convert from one type to another, sure, by explicitly invoking a function like atoi() or Integer.toString(), but the conversion is deliberate and so it is strongly typed. A variable containing a string (java.lang.String) cannot be compared against one containing a timestamp (java.util.Date) by accident. An Ion timestamp is a timestamp and can't be conflated with a string, although it can be converted to one.

Edit: The set of types that are built in, in conjunction with how those types are expressed in programming languages (e.g. timestamp as java.util.Date, decimal as java.math.BigDecimal, blob as byte[]), is why I'd call Ion strongly typed or richly typed in comparison to JSON. Specifically, scalar values that frequently appear in common programs can be expressed with distinctly typed scalar values in Ion. I don't know if there's a good formal definition. You could probably define a preorder on programming languages or data formats based simply on the number of distinct scalar or composite types (so in that sense, yes, it's the fact that Ion has more). However it goes beyond that subjectively. Subjectively it's about how often you have to, in practice, convert from one type to another in common tasks. There is no clear way to represent an arbitrary-precision decimal in JSON, or a byte array, or a timestamp -- so you must "compress" those types down into a single JSON type like string-of-some-format or array-of-number; and several different scalar types must all map to that same JSON type, which creates the risk of conflating values of different logical types but the same physical JSON type with each other. There's no obvious or built-in way to reconstruct the original type with fidelity. There's no self-describing path back from "1999-12-31T23:14:33.079-08:00" and "DEADBEEFBASE64" back to those original types.

I subjectively call JSON weakly typed because its types are not adequately to uniquely store common scalar data types that I work with in programs that I write. I call Ion strongly typed because it typically can. I acknowledged earlier that a data format would be even more strongly typed if it was capable of representing not just the type "integer", but "integer length meters". Ion does not have this kind of type built in, though its annotations feature could be used to describe that a particular integer value represents a length in meters.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#88

Post by jonhohle »

mattdeboard wrote: post #69
tantalor wrote: post #41

Earlier quoted context omitted.

> without requiring all consumers to have your schema Then how is the client supposed to handle the data? Guessing? > backwards compatible schemas > text and binary representations > type system > maps well to several languages Protos have all these. > S-Expressions Okay? Is that useful?

> Okay? Is that useful? I bet it's super useful if you have sexprs in your data.

It's interesting to have languages written for and in the same specification as your data.

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#89

Post by JonathonW »

conradev wrote: post #22
leef wrote: post #11

Finally! I've had to live the JSON nightmare since I left Amazon. Some of the benefits over JSON: * Real date type * Real binary type - no need to base64 encode * Real decimal type - invaluable when working with currency * Annotations - You can tag an Ion field in a map with an annotation that says, e.g. its compression ("csv", "snappy") or its serialized type ('com.example.Foo'). * Text and binary format * Symbol ta…

Sounds a lot like Apple's property list format, which shares almost everything you listed in common, except for annotations and symbol tables. Its binary format was introduced in 2002! Edit: Property lists only support integers up to 128 bits in size and double-precision floating point numbers. On top of those, Ion also supports infinite precision decimals.

Plists are nifty, but the text format's XML-based, which makes it too complex and too verbose to be a general-purpose alternative to something like JSON.

(plutil "supports" a json format, but it's not capable of expressing the complete feature set of the XML or binary formats.)

Re: Amazon open-sources Ion – a binary and text interchangable, typed JSON-superset

#90

Post by kazinator »

wyc wrote: post #44
kyllo wrote: post #25

Earlier quoted context omitted.

Real decimal type - invaluable when working with currency What does JavaScript do with this though, just cast it to a float?

I find that many financial technology companies opt to store currency as strings. The small overhead is typically well worth freedom from floating-point errors.

Any text format is a technically a string. Just because some numeric token has no double quotes around it doesn't mean it isn't a string (in that representation).

It's just that we can have some lexical rules that if that piece of text has the right kind of squiggly tail or whatever, it is always treated as a decimal float instead of every programmer working with it individually having to deal with an ad hoc string to decimal type conversion.

Post reply on HN