Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parsing JSON is a Minefield

seriot.ch

Post reply on HN
Print view View original

81–90 of 301 posts

Re: Parsing JSON is a Minefield

#81

Post by yes_or_gnome »

petre wrote: post #57
indexerror wrote: post #4

> In conclusion, JSON is not a data format you can rely on blindly. What does HN suggest for configuration files (to be written by a human essentially)? I am looking at YAML and TOML. My experience with JSON based config files was horrible.

YAML is equally horrible and the spec is an order of magnitude more complex. I wasted half an hour trying to spot an error in the ejabberd yaml config, only to find out something trivial was missing. At least JSON has braces even though it's not suitable for configuration files. By all means choose TOML or something else (even ini or java properties files) instead.

YAML has braces. In fact, it's a super set of JSON. Any YAML parser should be able to parse JSON encoded data with one exception. Block comments. Which is a bastardization of JSON (as mentioned in the article) so block comments shouldn't be a problem in most cases.

Re: Parsing JSON is a Minefield

#82

Post by lucb1e »

nulagrithom wrote: post #55
lucb1e wrote: post #23

Earlier quoted context omitted.

> Well, first and most obviously, if you are thinking of rolling your own JSON parser, stop and seek medical attention. Been there done that. (The medical attention, I mean.) Worked just fine. The article makes it sound extremely difficult, but 100% of the article is about edge cases that rarely happen with normal encoders and can often be ignored (e.g. who cares if you escape your tab character?). > consider what ha…

> edge cases that rarely happen with normal encoders and can often be ignored I've found that "edge cases that rarely happen" are typically 90% of my major headaches.

Certainly, edge cases are a source of pain. Luckily I mentioned that even if you ignore most of these edge cases (as a parser) the data will come out just fine.

Re: Parsing JSON is a Minefield

#83

Post by lucb1e »

s_q_b wrote: post #72
lucb1e wrote: post #23

Earlier quoted context omitted.

> Well, first and most obviously, if you are thinking of rolling your own JSON parser, stop and seek medical attention. Been there done that. (The medical attention, I mean.) Worked just fine. The article makes it sound extremely difficult, but 100% of the article is about edge cases that rarely happen with normal encoders and can often be ignored (e.g. who cares if you escape your tab character?). > consider what ha…

Yeah, the whole idea is a terrible practice, but it is used in industry surprisingly often. People think that PHP's session lock will save them, seeing the potential race condition but not the decode failure. The scenario is typically username-password-parameters passed as three variables via POST. The offending developer parses all three variables up front for simplicity: The session user is created from the POST us…

> Hint: this bug exists in the wild on a massively deployed framework. I'm working up a disclosure on that one now.

Alright you win. I'm curious to know which one it was after it's fixed!

I really didn't expect this to exist...

Re: Parsing JSON is a Minefield

#84

Post by [deleted] »

crdoconnor wrote: post #80
petre wrote: post #57

Earlier quoted context omitted.

YAML is equally horrible and the spec is an order of magnitude more complex. I wasted half an hour trying to spot an error in the ejabberd yaml config, only to find out something trivial was missing. At least JSON has braces even though it's not suitable for configuration files. By all means choose TOML or something else (even ini or java properties files) instead.

What was the trivial missing thing?

[deleted]

Re: Parsing JSON is a Minefield

#85

Post by marxidad »

herge wrote: post #35
mikeash wrote: post #32

Earlier quoted context omitted.

A parser should never crash on bad input. If it does, that's a serious bug that needs immediate attention, since that's at least a DoS vulnerability and quite likely something that could result in remote code execution. You definitely need to assume that the parser could fail , but that's different. Unless you're using "crash" in some way I'm not familiar with?

What about raise an exception?

Exceptions should only be used for exceptional cases. For a parser, bad input should be expected.

Re: Parsing JSON is a Minefield

#86

Post by mi100hael »

Tepix wrote: post #15
mi100hael wrote: post #3

> In conclusion, JSON is not a data format you can rely on blindly. That was definitely not my take-away from the article. More like "JSON is not a data format you can rely on blindly if you are using an esoteric edge-case and/or an alpha-stage parsing library." I haven't ever run into a single JSON issue that wasn't due to my own fat fingers or trying to serialize data that would have been better suited to something…

You are not considering a hostile environment like the internet where an attacker will use edge cases to get unforeseen results.

I couldn't really care less if someone POSTs some garbage JSON that results in them getting a 500 response. Better than someone POSTing an XML bomb and affecting other peoples' requests. Please enlighten me if you know of a serialization format with libraries for all common languages that lacks any gotchas or edge cases.

Re: Parsing JSON is a Minefield

#87

Post by [deleted] »

robert_tweed wrote: post #9
indexerror wrote: post #4

> In conclusion, JSON is not a data format you can rely on blindly. What does HN suggest for configuration files (to be written by a human essentially)? I am looking at YAML and TOML. My experience with JSON based config files was horrible.

Funnily enough, as I've been experimenting with Chef and trying to stick to JSON config files where allowed, I was again struck that (a) it's not a good choice for config files (b) it's an OK choice though (c) lots of people are using it anyway (d) nearly everyone that does so (including Chef) allows comments, so in reality are not actually using JSON at all. Point (d) is the important one. I really think we need a s…

[deleted]

Re: Parsing JSON is a Minefield

#88

Post by s_q_b »

mikeash wrote: post #78
s_q_b wrote: post #58

Earlier quoted context omitted.

Yes, exactly this. What I see a lot is initializing an object from a JSON parser which, when it receives malformed input, either halts execution outright or returns an empty object. In both cases you just want to make sure to handle the error appropriately per-language. Again, the main offender here is PHP, which makes this issue surprisingly easy to get wrong. (And add "==" to your lint checks! 99% of the time coder…

If you meant that it would throw or return an error on malformed input, I totally agree. You must always handle errors that your calls might return, and especially when those calls are given anything derived from outside input.

Yes, generally, and specifically that failure to handle this type of error can cause all sorts of plague, pestilence, and application compromise.

Re: Parsing JSON is a Minefield

#89

Post by djur »

edem wrote: post #79

JSON is the de facto standard when it comes to (un)serialising and exchanging data in web and mobile programming. I disagree. Take protobuf for example. You get schemas, data structures, and a parser in one package which is actually a lot smaller than JSON and compiles to nearly all the commonly used languages. Ever since I've started using it my life became so easier! If you don't want your data to be human readable…

JSON is still the de facto standard, regardless of whether it should be.

Re: Parsing JSON is a Minefield

#90

Post by mikeash »

s_q_b wrote: post #88
mikeash wrote: post #78

Earlier quoted context omitted.

If you meant that it would throw or return an error on malformed input, I totally agree. You must always handle errors that your calls might return, and especially when those calls are given anything derived from outside input.

Yes, generally, and specifically that failure to handle this type of error can cause all sorts of plague, pestilence, and application compromise.

Indeed, forgetting to check for errors is a great way to cause all sorts of terrible bugs. That's one reason I'm starting to become a fan of checked exceptions and result types.
Post reply on HN