Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Transit: JSON Data Interchange Format

github.com

Post reply on HN
Print view View original

91–100 of 124 posts

Re: Transit: JSON Data Interchange Format

#91

Post by tel »

brianberns wrote: post #79
gioele wrote: post #75

Earlier quoted context omitted.

> JSON has become the go-to schemaless format between languages, despite being verbose and having problems with typing. Transit aims to be a general-purpose successor to JSON here. XML has become the go-to schemaless format between languages, despite being verbose and having problems with typing. JSON aims to be a general-purpose successor to XML here.

Yes, except that XML has a schema (XSD).

That's an exterior thing. It's perfectly reasonable to talk about XML the schemaless markup language.

Re: Transit: JSON Data Interchange Format

#92

Post by dragonwriter »

hyperpallium wrote: post #62

> The extension mechanism is open, allowing programs using Transit to add new elements specific to their needs. Users of data formats without such facilities must rely on either schemas, convention, or context to convey elements not included in the base set, The extension mechanism is writing handlers in all languages communicated with, since its stated purpose is cross language value conveyance. In contrast, a schem…

> In contrast, a schema language allows extensions to be described once, in one language.

No, it doesn't. A schema language allows that to be done for the syntax, but requires the semantics to be implemented for each language. Schema languages often include, as part of their specifications, core types which must be supported; when an application restricts itself to these core types and types whose only important semantics are derived from them (e.g., restricted subsets of core types in most cases), then the fact that every full implementation that supports the core types will already have this work done for every language means no additional work is necessary. But that's not a product of a schema language preventing types to be implemented for all host languages, that's a result of the fact that a predefined set of core types accompanying the schema language means that all implementations are required to have already done the work of implementing the core types for all languages.

Re: Transit: JSON Data Interchange Format

#93

Post by tel »

jimktrains2 wrote: post #90
dragonwriter wrote: post #88

Earlier quoted context omitted.

> Floating point keys just seem like a terrible idea. I don't see why floating point (with a defined precision) keys are any worse fundamentally than any finite-domain scalar keys. The applications for which they are useful may be limited, and they may be the wrong choice for some uses, but that's true of anything.

> floating point (with a defined precision) If it's fixed-precision it's not floating point, it's basically an integer we interpret with a decimal point somewhere when we display it. If you didn't mean fixed-precision, then I'm not sure how actual floating points could sanely act as a hash key.

Even with a defined precision they're weird in that they "ought to" be modeling the reals and therefore lack computable equality.

Which makes it hard to write a well-defined hash.

Re: Transit: JSON Data Interchange Format

#94

Post by creshal »

prodigal_erik wrote: post #82
dec0dedab0de wrote: post #69

Earlier quoted context omitted.

I'm going to risk sounding thick here, but why would you need a strongly-typed data exchange format? I always thought the beauty of JSON was that it forced the sender to organize the data in a generic way, which allowed the receiver to interpret it however needed.

Without a strongly-typed data exchange format, you constantly have to write code that makes assumptions you hope the sender followed. Timestamps are ISO formatted. No, timestamps are seconds since 1970. No, they're floats including milliseconds. Blobs are base64. No, hex. Money is a string. No, it's a float. No, it's an int in micros. An empty array is different than an omitted field. No, they're the same. No, multip…

> Money is a string. No, it's a float. No, it's an int in micros

No, it's a decimal number with attribute-dependant accuracy settings, with at least three different accuracy levels used in the same object used for different purposes.

(Just something I came across today at work. Thankfully, my SOAP library handles that automatically…)

Re: Transit: JSON Data Interchange Format

#95

Post by tel »

togusa wrote: post #26
progx wrote: post #24

Earlier quoted context omitted.

Feel free to write a schema definition for json data (or use the existing). Why should this not be possible?

Well in this case, JSON schema doesn't specify any more fundamental types other than "number". On the receiving end of a wire or network contract, how do we pick a storage type for "number"? We can't because the constraints of the type are undefined. Ergo JSON schema isn't a strong schema language. Be explicit is really important when defining contracts. Type this in your address bar for an illustration: javascript:a…

Otoh, what's the difference between

    { "number": "1.0000000000000000000000000000001" }
along with

    { "number": { "type": "ModelReal", precision: 64 } }
versus

    1.0000000000000000000000000000001
along with
In each case you have text data given meaning by an external semantics enforced via a schema. The real and unavoidable downside is that JSON actually contains a really lousy primitive. It wouldn't be bad except practically every implementation of JSON automatically performs a lossy coercion to IEEE floats.

Re: Transit: JSON Data Interchange Format

#96

Post by [deleted] »

oneeyedpigeon wrote: post #58
togusa wrote: post #20

Earlier quoted context omitted.

Enterprise here. We like XML because it's explicit. { number: 1.0000000000000000000000000000001 } Try parsing the above JSON consistently in several languages without a consistent schema definition.

True, any problem domain that requires accuracy to 31 decimal places might not be best suited to JSON. I wonder how many decimal places would be required to represent the percentage of projects that would actually be affected by that...

[deleted]

Re: Transit: JSON Data Interchange Format

#97

Post by agentgt »

The main advantage it seems for Transit is that JSON is fast for many different clients. Otherwise I'm not sure I would ever use it for internal services considering there are so many other probably faster options (Protobuf,Avro,SBE ... etc).

Are folks using it for internal services?

Re: Transit: JSON Data Interchange Format

#98

Post by dragonwriter »

jimktrains2 wrote: post #90
dragonwriter wrote: post #88

Earlier quoted context omitted.

> Floating point keys just seem like a terrible idea. I don't see why floating point (with a defined precision) keys are any worse fundamentally than any finite-domain scalar keys. The applications for which they are useful may be limited, and they may be the wrong choice for some uses, but that's true of anything.

> floating point (with a defined precision) If it's fixed-precision it's not floating point, it's basically an integer we interpret with a decimal point somewhere when we display it. If you didn't mean fixed-precision, then I'm not sure how actual floating points could sanely act as a hash key.

> If it's fixed-precision it's not floating point

I'm not referring to fixed point, but fixed-size representation.

> If you didn't mean fixed-precision, then I'm not sure how actual floating points could sanely act as a hash key.

I don't see what the problem is. Certainly, there's problems if you do certain floating point calculations and blindly look for the result as a key in a mapping, but that's not a problem with floats as keys, its a problem with that particular usage pattern.

Re: Transit: JSON Data Interchange Format

#99

Post by waxjar »

falcolas wrote: post #73
waxjar wrote: post #50

Earlier quoted context omitted.

I don't think I would enjoy debugging a service that talks in a binary format.

That ship is at the edge of the harbor, sails raised, and helmed by HTTP2.0. Binary formats are becoming the default, not the exception.

True. Fortunately, we can be reasonably sure there will be multiple good tools to debug something talking over HTTP 2.0. With something like Transit, I'm not so sure.

Re: Transit: JSON Data Interchange Format

#100

Post by jimktrains2 »

dragonwriter wrote: post #98
jimktrains2 wrote: post #90

Earlier quoted context omitted.

> floating point (with a defined precision) If it's fixed-precision it's not floating point, it's basically an integer we interpret with a decimal point somewhere when we display it. If you didn't mean fixed-precision, then I'm not sure how actual floating points could sanely act as a hash key.

> If it's fixed-precision it's not floating point I'm not referring to fixed point, but fixed-size representation. > If you didn't mean fixed-precision, then I'm not sure how actual floating points could sanely act as a hash key. I don't see what the problem is. Certainly, there's problems if you do certain floating point calculations and blindly look for the result as a key in a mapping, but that's not a problem wit…

> I'm not referring to fixed point, but fixed-size representation.

I'm still not sure what you mean as any IEEE floating point (which in reality all anyone uses) is fixed-size. If you decide to truncate, round, do w/e to this value, you may still not get exactly the same answer for similar computations or even the same computation on different machines. There isn't a reason you can't (well, shouldn't) do float_val == 0.0 -- it won't work in all circumstances.

Please explain your method of making sure float_val == 0.0 while retaining the essence of being floating point (i.e. not using fixed-point).

Post reply on HN