For even more flexibility in JSON format there is SEN. https://news.ycombinator.com/item?id=26237048 talks about making JSON look good and introduces SEN at the end with optional commas and quotes. It also allows comments and converts back and forth with pure JSON with no loss of data. Comments are obvious lost though as JSON does not support comments.
JSON with Commas and Comments
211–220 of 251 posts
Re: JSON with Commas and Comments
#212Earlier quoted context omitted.
Imho, statically typed languages are the ones that benefit most from schema. The current schema version is 12 but the implementations for Go, Rust, C++ and Java are all listed as draft 7. None of them support codegen either, just validation, so not exactly compelling.
> The current schema version is 12 but the implementations for Go, Rust, C++ and Java are all listed as draft 7 It's actually 2020-12, which is two versions after Draft 7 (they shifted from Draft n to YYYY-MM after Draft 7, and since then have had 2019-09 and 2020-12.) And that's true of most languages, though there is some 2019-09 support. (It really doesn't help that there is also OpenAPI which baked in a variant—“…
OpenAPI 3.1 which was released recently, uses JSONSchema 2020-12 as the primary schema format. As a result, we can expect further consolidation of tooling, etc in the community.
Re: JSON with Commas and Comments
#213I just happen to be working on JSON parsing today. What a mess. People don't even follow what is already there. I've seen several 'JSON' data files like this: { } { } That isn't valid JSON, is it? The Qt JSON parser doesn't handle it.
That looks a bit like NDJSON, assuming that the space between the objects is actually a newline. There are libraries[1] that support parsing it and it's not too hard to do yourself, either. Some fairly popular projects use it to represent multiple responses in a single response body[2]. [1] http://ndjson.org/libraries.html [2] ElasticSearch uses it for msearch response bodys, for example.
It looks like it might be this:
https://jsonlines.org/
Which I hadn't heard of before.
Re: JSON with Commas and Comments
#214Earlier quoted context omitted.
Json does have schemas
The JSON schema ecosystem is a mess. json-schema.org is one thing, openapi is another (it uses an 'extended subset' of json-schema.org, which is another way of saying 'incompatible'), there’s also typeschema, and a bunch of other pet projects. There is no official JSON schema RFC, there is no official linking between json-schema.org and the JSON RFC. There is no push to make every language that supports JSON to also…
OpenAPI 3.1 which was released recently, uses JSONSchema 2020-12 as the primary schema format. The incompatibilities has been completely resolved as of today.
Re: JSON with Commas and Comments
#215Earlier quoted context omitted.
Petabytes of JSON formatted data flowing across the internet every second of every day disagree with your assessment.
And I suppose the vast amounts of XML (still!) flowing across the internet show there's no need for JSON at all , so the real mistake was even inventing JSON? (Also, as gets pointed out every time this topic pops up on HN, including multiple times on the submission already, the main demand isn't people wanting to add comments to the JS flowing across the wire, but to config files sitting on disk. Responding to people…
Crockford (and many others) found that the many flaws inherent to XML warranted a new specification, so JSON was born. If you have an issue with JSON, go create your own spec, don’t shit all over the perfectly capable, simple, elegant specification that is JSON.
Re: JSON with Commas and Comments
#216Earlier quoted context omitted.
As long as you set a limit on integer sizes (in most cases an arbitrary low limit should be fine) and don't expect any precision on float (each conversion between text and binary representation there is problematic) and don't use extensions (like comments or trailing commas) you should be quite fine with most JSON libraries. This won't cover all things, but a lot.
It's not a great standard if every program/library using it must explicitly state its limits for such basic things as number values. It's very easy to do JSON wrong, to not even be aware of doing it wrong, and do it wrong in a way that limits its interoperability (see: the default behaviour of the json library in Python). This is not theoretical either, just google for 'json nan github' to see the hundreds of product…
I agree that as a standard it isn't great. However the fact that it is so successful shows that the minimalistic design has benefits for adoption.
And regarding integer ranges: Any user has restrictions on top of the generic format. Some fields have to be present for the application to work, some fields have to be a string, others an array. Some integer has to be between 0 and 100, some array has to have 5 elements. Given that 99% of integers in practice fit in a signed int32 there is little problem (97% are probably 0, 1% are 1, 0.5% are -1 and only the rest other values ...) If you are on the edge you have to know and work-around ...
Re: JSON with Commas and Comments
#217Let's stop pretending JSON isn't popular because it lets us be loosey-goosey with our specifications. Writing specifications is hard and time consuming, and getting people to follow them is almost impossible. JSON is something we can (almost) all agree on for dumping loosely specified, human readable representations of data structures. It lets users and client application developers be lazy and not have to learn a ne…
I really want something that’s strongly typed but doesn’t require code generation like protobufs do. Yaml doesn’t do it for me. The closest I can get is putting the type guarantees in the database and using GraphQL.
And there's Amazon Ion too - https://amzn.github.io/ion-docs/
"Amazon Ion is a richly-typed, self-describing, hierarchical data serialization format offering interchangeable binary and text representations. The text format (a superset of JSON) is easy to read and author, supporting rapid prototyping. The binary representation is efficient to store, transmit, and skip-scan parse. The rich type system provides unambiguous semantics for long-term preservation of data which can survive multiple generations of software evolution."
Re: JSON with Commas and Comments
#218I just happen to be working on JSON parsing today. What a mess. People don't even follow what is already there. I've seen several 'JSON' data files like this: { } { } That isn't valid JSON, is it? The Qt JSON parser doesn't handle it.
That looks a bit like NDJSON, assuming that the space between the objects is actually a newline. There are libraries[1] that support parsing it and it's not too hard to do yourself, either. Some fairly popular projects use it to represent multiple responses in a single response body[2]. [1] http://ndjson.org/libraries.html [2] ElasticSearch uses it for msearch response bodys, for example.
Or JSON lines - whose website looks very similar to NDJSON's:
This format works great when using Amazon Athena (Presto) against log files written with one JSON object per line.
Re: JSON with Commas and Comments
#219Earlier quoted context omitted.
It's not a great standard if every program/library using it must explicitly state its limits for such basic things as number values. It's very easy to do JSON wrong, to not even be aware of doing it wrong, and do it wrong in a way that limits its interoperability (see: the default behaviour of the json library in Python). This is not theoretical either, just google for 'json nan github' to see the hundreds of product…
I agree that as a standard it isn't great. However the fact that it is so successful shows that the minimalistic design has benefits for adoption. And regarding integer ranges: Any user has restrictions on top of the generic format. Some fields have to be present for the application to work, some fields have to be a string, others an array. Some integer has to be between 0 and 100, some array has to have 5 elements.…
> Any user has restrictions on top of the generic format.
The problem is that you are not guaranteed to know, as a JSON library user, that the library has not mangled the numbers it received on the wire prior to your application receiving it - so you don't know if you having .a set to 42 is the result of 42 being sent over the wire, or an implementation dropping bits that are outside the range of you library's support. The RFC does not mandate what a JSON implementation should do in case of numbers outside its' support.
> Given that 99% of integers in practice fit in a signed int32 there is little problem.
Until you hit that 1%, and you find this out the hard way, and you have no way of solving it because you don't control the emitting side. Again, this has happened in practice to me, when a Python library was emitting large numbers as numbers (as the RFC permits), while the receiving side silently casted to 32-bit floats, losing data (as the RFC permits). Both sides are right per the spec. Technically, everything worked as per spec. Practically, the product was broken.
99% of the time you might be okay. But the 1% of edge cases makes it that you can never rely on JSON, which makes it a bad interchange format if you care about reliability and safety. You _can_ make it work if you severly limit yourself and are deeply aware of all the possible issues that using JSON has (and there's a lot more). Or you can just pick some other standard that solves these basic things for you (eg. Protobuf).
Re: JSON with Commas and Comments
#220Earlier quoted context omitted.
I agree that as a standard it isn't great. However the fact that it is so successful shows that the minimalistic design has benefits for adoption. And regarding integer ranges: Any user has restrictions on top of the generic format. Some fields have to be present for the application to work, some fields have to be a string, others an array. Some integer has to be between 0 and 100, some array has to have 5 elements.…
> Any user has restrictions on top of the generic format. The problem is that you are not guaranteed to know, as a JSON library user, that the library has not mangled the numbers it received on the wire prior to your application receiving it - so you don't know if you having .a set to 42 is the result of 42 being sent over the wire, or an implementation dropping bits that are outside the range of you library's suppor…
Any half serious library will be able to handle 42. If I leave the signed int32 range as a designer of the API I should be aware of the problem and either clearly document what I do or find some alternative. (Like making it a string with quotes)
Random side note: ECMAScript has no integer type, but only Number, which is a float, thus when dealing with such numbers in a JavaScript frontend you are in Problem Land anyways ... which again shows that boundaries have to be thought of, independently from the specification of the data exchange layer.