Live data from Hacker News

Parsing JSON is a Minefield

seriot.ch

61–70 of 257 posts

Re: Parsing JSON is a Minefield

#62

Earlier quoted context omitted.

HTML isn't XML. It's close, but it isn't. There's XHTML for that.

Just for the record - XML and HTML are both subsets of SGML, somewhat overlapping, but by no means coterminous with each other (at least until HTML 5 - I'm honestly not sure what it's relationship to SGML is). And, speaking from experience, the XML nay-sayers should largely be glad if they never had to deal with SGML :)

The main point of HTML5 is that it is not defined in terms of SGML but by it's own grammar which is in fact described by imperative algorith for parsing it (which also unambiguously specifies what should happen for notionally invalid inputs, AFAIK to the extent that for every byte stream there is exactly one resulting DOM tree).

Re: Parsing JSON is a Minefield

#63
post #34
post #20

Earlier quoted context omitted.

This is also widely considered a bad idea now. Making liberal consumers allows for sloppy producers. Over time this requires new consumers to conform to these sloppy producers to maintain compatibility. Just look at the clusterfuck that HTML5 has become. You need to have extremely deep pockets to enter that market.

Do you have a survey or other citation for it being a bad idea? I get that it enables bad behavior, per see. However, the idea of rejecting a customer/client because they did not form their request perfectly seems rather anti customer. Ideally, you'd both accept and correct. But that is the idea, just reworded.

The problem with this idea is that different consumers might have a different subset of what they accept and correct.

If some of those become dominant, produces might start depending on that behavior and it becomes a de facto standard. This is literally what has happened to HTML, but holds true for many other Internet protocols.

If you're looking for some external reading, I found at least this:

* https://tools.ietf.org/html/draft-thomson-postel-was-wrong

I think you'll find few protocol designers arguing _for_ the robustness principle these days.

Re: Parsing JSON is a Minefield

#64
post #34
post #20

Earlier quoted context omitted.

This is also widely considered a bad idea now. Making liberal consumers allows for sloppy producers. Over time this requires new consumers to conform to these sloppy producers to maintain compatibility. Just look at the clusterfuck that HTML5 has become. You need to have extremely deep pockets to enter that market.

Do you have a survey or other citation for it being a bad idea? I get that it enables bad behavior, per see. However, the idea of rejecting a customer/client because they did not form their request perfectly seems rather anti customer. Ideally, you'd both accept and correct. But that is the idea, just reworded.

The Harmful Consequences of Postel's Maxim - https://tools.ietf.org/html/draft-thomson-postel-was-wrong-0... (HN from 2015 https://news.ycombinator.com/item?id=9824638 )

Wrestling with Postel’s Law https://techblog.workiva.com/tech-blog/wrestling-postel’s-la...

Re: Parsing JSON is a Minefield

#65

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

Disagree. I can always make my JSON act like XML if I want to. When I'm following something like JSON API v1.1 I get a lot of the advantages that I'd get from XML with 99% less bloat. You want types? Go for it! There are even official typed JSON options out there. The security / parsing issues with XML alone are enough for me to rule it out. How many critical security issues are the result of libxml? Nokogiri / libxm…

> […] it's dead simple to parse […]

I've heard it's a minefield.[0]

[0]: http://seriot.ch/parsing_json.php

Re: Parsing JSON is a Minefield

#66

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

XML parsing is notably an even larger minefield: https://www.owasp.org/index.php/XML_Security_Cheat_Sheet

I've written JSON parsers to replace platform specific JSON parsers with bug-for-bug (or at the very least misfeature-for-misfeature) parity to port code without breaking it, without too much going terribly wrong. I wouldn't even try to attempt the same for XML.

Generating a useful conservative subset of JSON that most/all JSON serializers will accept hasn't been that hard in practice IME (no trailing commas, escape all unicode, don't assume >double precision/range scalars, etc.), but I still haven't figured out how to do the same for some XML serializers (failing to serialize because it lacks 'extra' annotation tags in some cases, failing to serialize because it doesn't ignore 'extra' annotation tags in other cases...)

Re: Parsing JSON is a Minefield

#67
post #21

I think the idea of humans sharing a language with computers is problematic at a fundamental level. (the whole $dataformat "easy to read for humans") It becomes a source of never ending lose-lose compromises where the more points you give to human convenience the more points you take away from machine convenience and vice versa. Then you end up having to "settle" for something in between that is just ambiguous and pr…

>> I don't hear people wanting a human readable text representation of their audio, video or images. This is, in fact, a huge concern for people who think about accessibility.

Generally the way we address it is not to make the formats themselves human-readable; that's a fool's errand. Instead we create tools to transform them into human-readable formats on demand. See also the protobuf toolchain.

Re: Parsing JSON is a Minefield

#68

Before JSON, XML and standard binary formats, there were just CSV/TSV and random binary formats which was a bigger minefield. Simply exchanging data was a project in itself. At least JSON and XML are text based when it comes to data exchange. Back in the day before APIs that needed to exchange data cleanly, without JSON/XML, exchanging data was not only a minefield but one with constant carpet bombing. The fact that…

You forgot BER ;)

Re: Parsing JSON is a Minefield

#69
post #39

i wish there was a chance for EDN[1] to replace JSON. it's a shame the industry defaulted to a subset of javascript as a data notation format considering all it's shortcomings =/ yeah, i get it, "but it has native support in all browsers" is a valid argument, i just wish it wasn't. [1] https://github.com/edn-format/edn

In what regard EDN is "better" than JSON? the point of the post was the RFC specification is not tight and there are corner cases. I don't see any rigor in the given link either..

Re: Parsing JSON is a Minefield

#70

Earlier quoted context omitted.

Yep, the application to text documents is valid in my eyes, as well. Although there are lighter weight and/or more extensible approaches, like TeX. (update, clarificaton: I mean just the markup syntax, not the compuational model)

I like TeX for producing documents. But I'd take XML over TeX if I had to parse the markup myself, outside of the TeX toolchain. Any nontrivial TeX document is built out of a pile of macros, so you need to implement a TeX-compatible macro expander to parse it. And at least with XML there are solid libraries, while the state of TeX-parsing libraries outside of TeX itself is pretty poor. I think Haskell is the only lan…

It doesn't need to be Tex-compatible. My point was just that the syntax is lighter weight and might be preferable for some applications.
Post reply on HN