Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parsing JSON is a Minefield

seriot.ch

Post reply on HN
Print view View original

11–20 of 257 posts

Re: Parsing JSON is a Minefield

#11

Post by codahale »

eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

XML parsing is notably an even larger minefield: https://www.owasp.org/index.php/XML_Security_Cheat_Sheet

Re: Parsing JSON is a Minefield

#12

Post by leothelocust »

JasonFruit wrote: post #2

This is interesting and important in one way: anything poorly specified will eventually cause a problem for someone, somewhere. That being said, my first response was to complete the title, ". . . yet it remains useful and nearly trouble-free in practice." There's a lot of, "You know what I mean!" in the JSON definition, but in most cases, we really do know what Crockford means.

Fully agree. You can cause parsing issues, but you can also... not.

If you are creating a JSON response from your own API, you control the JSON output.

Unless you are crafting JSON from scratch I doubt anyone runs into the issues mentioned in the OP.

Re: Parsing JSON is a Minefield

#13

Post by kcolford »

This brings to mind the old internet motto (someone correct me on the actual source): "be liberal in what you accept, and be conservative in what you send".

JSON is pretty clear on what certain things should mean, strings are Unicode plus escape sequences, objects map keys to values, arrays are ordered collections of values, the whole serialized payload should be Unicode, etc. Even those things can be relaxed further. This IMHO is what makes JSON so robust on the internet and the perfect choice for a non-binary communication protocol.

Re: Parsing JSON is a Minefield

#14

Post by rmrfrmrf »

eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

If you think XML doesn't suffer from all the same issues, you haven't used it enough. I'd use protobuf for something that needs stict serialization and parsing.

Re: Parsing JSON is a Minefield

#15

Post by 0xcde4c3db »

kcolford wrote: post #13

This brings to mind the old internet motto (someone correct me on the actual source): "be liberal in what you accept, and be conservative in what you send". JSON is pretty clear on what certain things should mean, strings are Unicode plus escape sequences, objects map keys to values, arrays are ordered collections of values, the whole serialized payload should be Unicode, etc. Even those things can be relaxed further…

> "be liberal in what you accept, and be conservative in what you send"

This is commonly known as Postel's Law, and comes from one of the TCP RFCs [1].

[1] https://en.wikipedia.org/wiki/Robustness_principle

Re: Parsing JSON is a Minefield

#16

Post by jwilk »

kcolford wrote: post #13

This brings to mind the old internet motto (someone correct me on the actual source): "be liberal in what you accept, and be conservative in what you send". JSON is pretty clear on what certain things should mean, strings are Unicode plus escape sequences, objects map keys to values, arrays are ordered collections of values, the whole serialized payload should be Unicode, etc. Even those things can be relaxed further…

https://tools.ietf.org/html/rfc761#section-2.10

Re: Parsing JSON is a Minefield

#17

Post by Someone »

JasonFruit wrote: post #2

This is interesting and important in one way: anything poorly specified will eventually cause a problem for someone, somewhere. That being said, my first response was to complete the title, ". . . yet it remains useful and nearly trouble-free in practice." There's a lot of, "You know what I mean!" in the JSON definition, but in most cases, we really do know what Crockford means.

If your API takes json input, some of those issues are potential security or DoS issues.

For example, if you validate your json in your web front-end (EDIT: I used the wrong term. What I meant here is the server-side process that’s in front of your database) and then pass the string received to your json-aware database, you’re likely using two json implementations that may have different ideas about what constitutes valid json.

For example, a caller might pass in a dictionary with duplicate key names, and the two parsers might each drop a different one, or one might see json where the other sees a comment.

Re: Parsing JSON is a Minefield

#18

Post by inglor »

> For instance, RFC 8259 mentions that a design goal of JSON was to be "a subset of JavaScript", but it's actually not.

Actually, it's really close https://github.com/tc39/proposal-json-superset

This is a stage 3 proposal likely to make it to the next version of the spec. At which point JSON would truly be a subset of JavaScript.

Re: Parsing JSON is a Minefield

#19

Post by 3pt14159 »

eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

Disagree.

I can always make my JSON act like XML if I want to. When I'm following something like JSON API v1.1 I get a lot of the advantages that I'd get from XML with 99% less bloat. You want types? Go for it! There are even official typed JSON options out there. The security / parsing issues with XML alone are enough for me to rule it out.

How many critical security issues are the result of libxml? Nokogiri / libxml accounts for 50% of my emergency patches to my servers. ONE RUBY GEM is the result of half of my security headaches. That's insane. I only put up with it because other people choose to use XML and I want their data.

How many issues are the result of browsers having to deal with broken HTML (a form of XML)?

JSON isn't perfect, and I wouldn't use it absolutely everywhere, but it's dead simple to parse[0], readable without the whitespace issues of YAML, and I can't think of one place I'd use XML over it.

[0] http://json.org/

Re: Parsing JSON is a Minefield

#20

Post by bhldr »

0xcde4c3db wrote: post #15
kcolford wrote: post #13

This brings to mind the old internet motto (someone correct me on the actual source): "be liberal in what you accept, and be conservative in what you send". JSON is pretty clear on what certain things should mean, strings are Unicode plus escape sequences, objects map keys to values, arrays are ordered collections of values, the whole serialized payload should be Unicode, etc. Even those things can be relaxed further…

> "be liberal in what you accept, and be conservative in what you send" This is commonly known as Postel's Law, and comes from one of the TCP RFCs [1]. [1] https://en.wikipedia.org/wiki/Robustness_principle

This is also widely considered a bad idea now. Making liberal consumers allows for sloppy producers. Over time this requires new consumers to conform to these sloppy producers to maintain compatibility.

Just look at the clusterfuck that HTML5 has become. You need to have extremely deep pockets to enter that market.

Post reply on HN