Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parsing JSON is a Minefield

seriot.ch

Post reply on HN
Print view View original

141–150 of 301 posts

Re: Parsing JSON is a Minefield

#141

Post by ben_jones »

inimino wrote: post #34
s_q_b wrote: post #14

Well, first and most obviously, if you are thinking of rolling your own JSON parser, stop and seek medical attention. Secondly, assume that parsing your input will crash, so catch the error and have your application fail gracefully. This is the number one security issue I encounter in "security audited" PHP. (The second being the "==" vs. "===" debacle that is PHP comparison.) As one example, consider what happens wh…

Parsing HTML is literally orders of magnitude more complex. It is pretty rare to need to parse JSON yourself (what environment doesn't have that available?) but it isn't that difficult. It's a simple language.

Silent moment for those of us using niche languages to meet production requirements in environments that do not allow third-party code and do not have JSON parsing in the std lib...

Re: Parsing JSON is a Minefield

#142

Post by grayrest »

peatmoss wrote: post #61

What ever happened with EDN (pronounced "eden") from the Clojure people? https://clojure.github.io/clojure/clojure.edn-api.html https://github.com/edn-format/edn I always thought that seemed like a nice alternative data format to JSON. Anyone using this it in the wild?

It got supplanted by Transit [0] as an interchange format. Both are only really used within the Clojure community. I use Transit for internal-facing APIs.

[0] https://github.com/cognitect/transit-format

Re: Parsing JSON is a Minefield

#143

Post by pbowyer »

s_q_b wrote: post #72
lucb1e wrote: post #23

Earlier quoted context omitted.

> Well, first and most obviously, if you are thinking of rolling your own JSON parser, stop and seek medical attention. Been there done that. (The medical attention, I mean.) Worked just fine. The article makes it sound extremely difficult, but 100% of the article is about edge cases that rarely happen with normal encoders and can often be ignored (e.g. who cares if you escape your tab character?). > consider what ha…

Yeah, the whole idea is a terrible practice, but it is used in industry surprisingly often. People think that PHP's session lock will save them, seeing the potential race condition but not the decode failure. The scenario is typically username-password-parameters passed as three variables via POST. The offending developer parses all three variables up front for simplicity: The session user is created from the POST us…

Thanks for the extra explanation - I too doubted this could exist (surely the session existing alone isn't enough to assume a user is authenticated - need a variable set & checked every request?) but I see that it could.

I look forward to the disclosure to understand more about this - and check that the frameworks I use don't have this problem.

Re: Parsing JSON is a Minefield

#144

Post by romaniv »

mi100hael wrote: post #3

> In conclusion, JSON is not a data format you can rely on blindly. That was definitely not my take-away from the article. More like "JSON is not a data format you can rely on blindly if you are using an esoteric edge-case and/or an alpha-stage parsing library." I haven't ever run into a single JSON issue that wasn't due to my own fat fingers or trying to serialize data that would have been better suited to something…

If there are two servers/services that communicate via JSON and they use different parsers, these types of issues can lead to rather nasty problems. Even if both parties fail gracefully on their own.

This gets even worse if your software is an integration layer between two services you do not control.

Re: Parsing JSON is a Minefield

#145

Post by oblio »

ben_jones wrote: post #141
inimino wrote: post #34

Earlier quoted context omitted.

Parsing HTML is literally orders of magnitude more complex. It is pretty rare to need to parse JSON yourself (what environment doesn't have that available?) but it isn't that difficult. It's a simple language.

Silent moment for those of us using niche languages to meet production requirements in environments that do not allow third-party code and do not have JSON parsing in the std lib...

If your language doesn't come with JSON in the stdlib, you're really on the cutting edge. Or using a language meant for embedding :)

Re: Parsing JSON is a Minefield

#146

Post by oblio »

rch wrote: post #38
wtetzner wrote: post #27

Earlier quoted context omitted.

> Setting the session flag for "this user is logged in" before checking (or even decoding!) the password seems rather backwards to me. Yeah, that seems like a problem regardless of whether or not you're parsing JSON.

Probably a symptom of the PHP multiverse: anything that can happen, has happened.

And will continue to happen in a Wordpress setup somewhere.

Re: Parsing JSON is a Minefield

#147

Post by eridius »

Speaking as someone who wrote a JSON parser, this article and the accompanying test suite looks to be very valuable, and I will be adding this test suite to my parser's tests shortly.

That said, since my parser is a pure-Swift parser, I'm kind of bummed that the author didn't include it already, but instead chose to include an apparently buggy parser by Big Nerd Ranch instead. My parser is https://github.com/postmates/PMJSON

Re: Parsing JSON is a Minefield

#148

Post by sirn »

indexerror wrote: post #4

> In conclusion, JSON is not a data format you can rely on blindly. What does HN suggest for configuration files (to be written by a human essentially)? I am looking at YAML and TOML. My experience with JSON based config files was horrible.

If you decide to use YAML, make sure to check out Strict YAML[1] and its FAQ[2].

[1]: https://github.com/crdoconnor/strictyaml

[2]: https://github.com/crdoconnor/strictyaml/blob/master/FAQ.rst...

Re: Parsing JSON is a Minefield

#149

Post by mSparks »

inimino wrote: post #98
mSparks wrote: post #56

Earlier quoted context omitted.

How does json "not support comments"? {"comment":"default values for this object"}

For one thing, not every place where you might want a comment happens to be in an object.

Like where? Who for? For what purpose?

I cant think of a single example that this would be useful.

Re: Parsing JSON is a Minefield

#150

Post by stevetrewick »

marxidad wrote: post #85
herge wrote: post #35

Earlier quoted context omitted.

What about raise an exception?

Exceptions should only be used for exceptional cases. For a parser, bad input should be expected.

Parsing is often deeply recursive and in some languages, throwing an exception will automatically unwind the stack until it finds a handler. As well as explicitly giving you a good path and a bad path (as someone pointed out upthread) this can (again, in some languages) save a ton of repeated special case code in your parsing routines.

Some languages or libraries are explicitly designed to use exceptions for flow control some strongly discourage this. Apple's ObjC error handling guide for example contains this stricture but also calls out parsing as an example of when you should do it.

C.A.R 'null pointer' Hoare considered exceptions a blight on the earth that would lead to us accidentally nuking ourselves, so there's a spectrum of opinions available here.

Post reply on HN