Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parsing JSON is a Minefield

seriot.ch

Post reply on HN
Print view View original

71–80 of 257 posts

Re: Parsing JSON is a Minefield

#71

Post by bufferoverflow »

JasonFruit wrote: post #2

This is interesting and important in one way: anything poorly specified will eventually cause a problem for someone, somewhere. That being said, my first response was to complete the title, ". . . yet it remains useful and nearly trouble-free in practice." There's a lot of, "You know what I mean!" in the JSON definition, but in most cases, we really do know what Crockford means.

> anything poorly specified

I thought JSON was specified quite clearly.

http://json.org/

There are no limits of the loopy things (the number of consecutive digits in numbers), but I don't consider that a weakness of the standard.

Most of the tests that I see do pass completely invalid JSON.

http://seriot.ch/json/pruned_results.png

Re: Parsing JSON is a Minefield

#72

Post by zmix »

3pt14159 wrote: post #19
eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

Disagree. I can always make my JSON act like XML if I want to. When I'm following something like JSON API v1.1 I get a lot of the advantages that I'd get from XML with 99% less bloat. You want types? Go for it! There are even official typed JSON options out there. The security / parsing issues with XML alone are enough for me to rule it out. How many critical security issues are the result of libxml? Nokogiri / libxm…

> I can always make my JSON act like XML if I want to.

You can make your JSON have two identities at the same time: Document and data structure?

Re: Parsing JSON is a Minefield

#73

Post by dfox »

teddyh wrote: post #48
eponeponepon wrote: post #41

Earlier quoted context omitted.

Just for the record - XML and HTML are both subsets of SGML, somewhat overlapping, but by no means coterminous with each other (at least until HTML 5 - I'm honestly not sure what it's relationship to SGML is). And, speaking from experience, the XML nay-sayers should largely be glad if they never had to deal with SGML :)

HTML pretended to be a subset of SGML, but never really was, and the illusion quickly dispersed as time went on, since HTML was strictly pragmatic and ran in resource-constrained environments (the desktop), while SGML was academic, largely theoretical, and ran on servers, analyzing text. XML, on the other hand, was more of a back-formation – a generalization of HTML; it was not, as I understand it, directly related t…

XML was intended as subset of SGML that can be meaningfully parsed without knowing DTD of document in question, which involves removing a lot of weird SGML features and constraining others. Formally XML is not SGML subset as there are some unimportant and some quite critical incompatible details.

Re: Parsing JSON is a Minefield

#75

Post by dfox »

dboreham wrote: post #68
drawkbox wrote: post #56

Before JSON, XML and standard binary formats, there were just CSV/TSV and random binary formats which was a bigger minefield. Simply exchanging data was a project in itself. At least JSON and XML are text based when it comes to data exchange. Back in the day before APIs that needed to exchange data cleanly, without JSON/XML, exchanging data was not only a minefield but one with constant carpet bombing. The fact that…

You forgot BER ;)

And XDR, although in it's case it is slightly unclear whether it is standardized or random binary format ;)

Re: Parsing JSON is a Minefield

#76

Post by taeric »

treve wrote: post #63
taeric wrote: post #34

Earlier quoted context omitted.

Do you have a survey or other citation for it being a bad idea? I get that it enables bad behavior, per see. However, the idea of rejecting a customer/client because they did not form their request perfectly seems rather anti customer. Ideally, you'd both accept and correct. But that is the idea, just reworded.

The problem with this idea is that different consumers might have a different subset of what they accept and correct. If some of those become dominant, produces might start depending on that behavior and it becomes a de facto standard. This is literally what has happened to HTML, but holds true for many other Internet protocols. If you're looking for some external reading, I found at least this: * https://tools.ietf.…

You'll also find few protocol designers designing anything as robust as the old protocols. :)

I mean, don't go out of your way to under specify input. But relatively nobody is going back to the heavy schema of xml over simple json. Even if they probably should.

I feel this is an anti fragile position. Try not to encourage poor input. But more importantly, be resilient to it. Not dismissive of it.

Re: Parsing JSON is a Minefield

#77

Post by Boulth »

taeric wrote: post #34
bhldr wrote: post #20

Earlier quoted context omitted.

This is also widely considered a bad idea now. Making liberal consumers allows for sloppy producers. Over time this requires new consumers to conform to these sloppy producers to maintain compatibility. Just look at the clusterfuck that HTML5 has become. You need to have extremely deep pockets to enter that market.

Do you have a survey or other citation for it being a bad idea? I get that it enables bad behavior, per see. However, the idea of rejecting a customer/client because they did not form their request perfectly seems rather anti customer. Ideally, you'd both accept and correct. But that is the idea, just reworded.

Consumer is interested in fulfilling their need so they will fix their request so that it gets processed.

Re: Parsing JSON is a Minefield

#78

Post by ris »

3pt14159 wrote: post #19
eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

Disagree. I can always make my JSON act like XML if I want to. When I'm following something like JSON API v1.1 I get a lot of the advantages that I'd get from XML with 99% less bloat. You want types? Go for it! There are even official typed JSON options out there. The security / parsing issues with XML alone are enough for me to rule it out. How many critical security issues are the result of libxml? Nokogiri / libxm…

> the whitespace issues of YAML

It's not just whitespace issues yaml has - try storing ISO two letter country codes as values and as soon as you get to norway, you've got a boolean.

There are many things which are deceptively "simple" where that actually means "I haven't thought about this very much".

Re: Parsing JSON is a Minefield

#79

Post by izacus »

skywhopper wrote: post #52

While this is true of JSON, it's also true of any other non-trivial serialization and/or encoding format. The main lessons to learn here are that: 1) implementation matters 2) "simple" specs never really are It's definitely important to have documents like this one that explore the edge cases and the differences between implementations, but you can replace "JSON" in the introductory paragraph with any other serializa…

How do Protocol Buffers (which I see used quite alot in similar environments as JSON) compare? Anyone has experience in the format?

Re: Parsing JSON is a Minefield

#80

Post by [deleted] »

paradite wrote: post #58
Someone wrote: post #17

Earlier quoted context omitted.

If your API takes json input, some of those issues are potential security or DoS issues. For example, if you validate your json in your web front-end (EDIT: I used the wrong term. What I meant here is the server-side process that’s in front of your database) and then pass the string received to your json-aware database, you’re likely using two json implementations that may have different ideas about what constitutes…

I think it is rather common sense to do data validation on backend instead of frontend. What matters is that backend always acts as the source of truth, it doesn't really matter if frontend and backend are inconsistent as long as we know that backend data is correct.

[deleted]
Post reply on HN