Earlier quoted context omitted.
No, this is not true of many reasonable formats. You don't have to make an obtusely nontrivial format to encode the data JSON does.
Most of the things they test are true of any text-based format, and many of them are true of any serialisation format. E.g. 100000 opening brackets. You could do the same in XML for example and I expect many parsers would fail.
Parsing JSON is a Minefield
171–180 of 257 posts
Re: Parsing JSON is a Minefield
#172Reminds me of some JSON I got from an API. It was always malformed and I always wrote the dev that he should fix it. He always did, but every new endpoint was malformed again. One day I looked at the code and it was full of string concatinations of DB results...
select '{ "user": { "name": "' || u.name || '", "email": ' || u.email || '" } }' as json from users u;
oh my.Re: Parsing JSON is a Minefield
#173Earlier quoted context omitted.
I meant that grammar policing does little to help the exchange of information. Feelings aside.
A lack of enforced language standards does end up producing a language full of difficult-to-learn inconsistencies, though.
Edit: I'm specifically going off evidence of teaching my kids. They have basically picked up language completely by talking to us. Even pronouns, adjectives, adverbs, etc. What they have not learned, is the reasons some words are used when another could have worked.
Re: Parsing JSON is a Minefield
#174Earlier quoted context omitted.
You'll also find few protocol designers designing anything as robust as the old protocols. :) I mean, don't go out of your way to under specify input. But relatively nobody is going back to the heavy schema of xml over simple json. Even if they probably should. I feel this is an anti fragile position. Try not to encourage poor input. But more importantly, be resilient to it. Not dismissive of it.
Robust as the old protocols? The early TCP protocols are very much underspecified. Maybe you have something else in mind.
I've just gotten weary of so many replacement protocols that get dreamed up and go nowhere. Often because they didn't actually learn all of the lessons from predecessors.
Re: Parsing JSON is a Minefield
#175Parsing JSON is not a minefield. It is technically trivial and pretty secure. Compared to other specs it's not that bad, but of course there are still some security concerns, esp. in the last two JSON RFC updates, which made it worse and not better. But most other commonly uses transport formats are much worse, and much harder to parse. Start reading at http://search.cpan.org/~rurban/Cpanel-JSON-XS-4.02/XS.pm#RFC...
Re: Parsing JSON is a Minefield
#176Earlier quoted context omitted.
>> I don't hear people wanting a human readable text representation of their audio, video or images. This is, in fact, a huge concern for people who think about accessibility.
Ok but what I'm talking about is a little more specific. Talking about "human readability" of JSON and XML is a little bit like talking about human readability of JPEG or MP3. Chasing after it creates a lot of problems. Formats like JSON and XML often carry lots of textual information so it's tempting to want them to be like "just like text but with some extra stuff" but that creates its own problems. So it would be…
Re: Parsing JSON is a Minefield
#177I think the idea of humans sharing a language with computers is problematic at a fundamental level. (the whole $dataformat "easy to read for humans") It becomes a source of never ending lose-lose compromises where the more points you give to human convenience the more points you take away from machine convenience and vice versa. Then you end up having to "settle" for something in between that is just ambiguous and pr…
Re: Parsing JSON is a Minefield
#178Reminds me of some JSON I got from an API. It was always malformed and I always wrote the dev that he should fix it. He always did, but every new endpoint was malformed again. One day I looked at the code and it was full of string concatinations of DB results...
select '{ "user": { "name": "' || u.name || '", "email": ' || u.email || '" } }' as json from users u; oh my.
Re: Parsing JSON is a Minefield
#179Earlier quoted context omitted.
If your API takes json input, some of those issues are potential security or DoS issues. For example, if you validate your json in your web front-end (EDIT: I used the wrong term. What I meant here is the server-side process that’s in front of your database) and then pass the string received to your json-aware database, you’re likely using two json implementations that may have different ideas about what constitutes…
Reminds me of last years CouchDB bug (CVE-2017-12635) which was caused by two JSON parsers disagreeing on duplicate keys: here it was possible to add a second key with user roles, allowing a user to give admin rights to itself. JSON parser issues are real.
By representing fields with enums or proper types you get some constraints on values as well, eg: If a value is really an integer field then your type can declare it as Int and deserialization will smash it into that shape or throw an error, but you don't end up with indeterminate or nonsense values.
This can be even more important for UUIDs, Dates, and other extremely common types that have no native JSON representation, nor even any agreed-upon consensus around them.
You get less help from the language with dynamic languages like Python but you can certainly accomplish the same thing with some minimal extra work. Or perhaps it would be more accurate to say languages like Python offer easy shortcuts that you shouldn't take.
In any case I highly recommend this technique for enforcing basic sanitization of data. The other is to use fuzzing (AFL or libFuzzer).
Re: Parsing JSON is a Minefield
#180Earlier quoted context omitted.
> Before JSON, XML and standard binary formats, there were just CSV/TSV and random binary formats which was a bigger minefield. S-expressions predate both, are simpler to parse than either, are more legible than both and are cheaper than either. Here's a JSON example from http://json.org/example.html : { "glossary": { "title": "example glossary", "GlossDiv": { "title": "S", "GlossList": { "GlossEntry": { "ID": "SGML"…
The S-expression has cleaner whitespace and field names than the JSON, which makes it harder to make an apples-to-apples comparison. But the biggest problem with that S-expression is that I don't know how to parse it. Is SGML a symbol, identifier, a quoteless string? How do I know when parsing the 'entry' field that what follows is going to be a list of key/value pairs without parsing the whole expression? Is 'see-al…
It's a sequence of bytes — a string, if you like.
> How do I know when parsing the 'entry' field that what follows is going to be a list of key/value pairs without parsing the whole expression?
You wouldn't, and as a parser you wouldn't need to. The thing which accepts the parsed lists of byte-sequences would need to know what to do with whatever it's given, but that's the same issue as is faced by something which accepts JSON.
> Is 'see-also GML XML' parsed as a list?
(see-also GML XML) is a list.
> How do we distinguish between single element lists and scalars?
'(single-element-list)' is a single-element list; 'scalar' is a scalar. Just like '["single-element-list"]' & '"scalar"' in JSON.
> Is it possible to express a list at the top level, like JSON allows?
That whole expression is a list at top level.
> How do you express a boolean, or null?
The same way that you represent a movie, a post or an integer: by applying some sort of meaning to a sequence of bytes.
> They do not solve questions of maximum levels of nesting.
They don't solve the problem of finite resources, no. It'll always be possible for someone to send one more data than one can possibly process.
> They have the same potential pitfalls with whitespace.
No, they don't, because Ron Rivest's canonical S-expression spec indicates exactly what is & is not whitespace.
> They have exactly the same problems with parsing strings and numbers.
No they don't, because they don't really have either strings or numbers: they have lists an byte-sequences. Anything else is up to the application which uses them — just like any higher meaning of JSON is up to the application which uses it.
> They have the same problem with duplicated keys.
No, they don't — because they don't have keys.
> My point is that in order to match what JSON can do, you have to create rules for interpreting the S-expressions, and those rules are the hard part.
My point is that JSON doesn't — and can't — create all the necessary rules, and that trying to do so is a mistake, because applications do not have mutually-compatible interpretations of data. One application may treat JSON numbers as 64-bit integers, another as 32-bit floats. One application may need to hash object cryptographically, and thus specify an ordering for object properties; another may not care. Every useful application will need to do more than just parse JSON into the equivalent data structure in memory: it needs to validate it & then work with it, which almost certainly means converting that JSON-like data structure into an application-specific data structure.
The key, IMHO, is to punt on specifying all of that for everyone for all time and instead to let each application specify its protocol as necessary. The reason to use S-expressions for that is that they are structured and capable of representing anything.
Ultimately, we can do more by doing less. JSON is seductive, but it'll ultimately leave one disappointed. It does a lot, but not enough. S-expressions do enough to let you do the rest.