Live data from Hacker News

Parsing JSON is a Minefield

seriot.ch

91–100 of 301 posts

Re: Parsing JSON is a Minefield

#91
post #29

> In conclusion, JSON is not a data format you can rely on blindly. What does HN suggest for configuration files (to be written by a human essentially)? I am looking at YAML and TOML. My experience with JSON based config files was horrible.

Lua was written for exactly this purpose, and I personally enjoy writing with it, so it would be my first choice in most cases.

What purpose? Writing config files? Lua is a programming language (a nice one too). It's code. Code should not be used for config files, nor for data serialization because once you eval it you are executing it.

Re: Parsing JSON is a Minefield

#92
post #35

Earlier quoted context omitted.

What about raise an exception?

Exceptions should only be used for exceptional cases. For a parser, bad input should be expected.

The JavaScript JSON parser throws an exception on invalid input. A benefit of this is that there is only one code path to handle any kind of failure, and another is that unhandled parse failures lead to a hard stop, which is a good default.

Re: Parsing JSON is a Minefield

#93
post #35

Earlier quoted context omitted.

What about raise an exception?

Exceptions should only be used for exceptional cases. For a parser, bad input should be expected.

You say that as fact but you must know it is a matter of opinion: I would say you should match the language idioms. For example, Python iterators and generators work by raising/throwing when there are no more items: it is fully expected and will always happen when you write a for-in loop.

Re: Parsing JSON is a Minefield

#94
post #32
post #14

Well, first and most obviously, if you are thinking of rolling your own JSON parser, stop and seek medical attention. Secondly, assume that parsing your input will crash, so catch the error and have your application fail gracefully. This is the number one security issue I encounter in "security audited" PHP. (The second being the "==" vs. "===" debacle that is PHP comparison.) As one example, consider what happens wh…

A parser should never crash on bad input. If it does, that's a serious bug that needs immediate attention, since that's at least a DoS vulnerability and quite likely something that could result in remote code execution. You definitely need to assume that the parser could fail , but that's different. Unless you're using "crash" in some way I'm not familiar with?

Recursion kills.

Re: Parsing JSON is a Minefield

#95
post #64

Now the mess that is called JavaScript dates has crept into any system imaginable in the world. I can understand we needed to go for the lowest denominator but Crockford's card really could cram in another line with a date time string format.

Just use epoch or ISO1601.

Re: Parsing JSON is a Minefield

#96
post #48

Wrote my own JSON parser ( https://github.com/MJPA/SimpleJSON ) a while ago... not sure how it's a minefield unless I'm missing something?

Did you read the post? The JSON standard(s) are very simple. But in practice this simplicity leads to a lot of edge cases. Very deeply nested structures, numbers that run on to infinity. The point of this post is testing various parsers against these malicious structures. See this image: http://seriot.ch/json/pruned_results.png

"In conclusion, JSON is not a data format you can rely on blindly." - that suggests the format is bad when it's highlighting problems with the parsers. I see it like saying "Plain text is a bad format because notepad bails on large files"

Re: Parsing JSON is a Minefield

#97
post #95
post #64

Now the mess that is called JavaScript dates has crept into any system imaginable in the world. I can understand we needed to go for the lowest denominator but Crockford's card really could cram in another line with a date time string format.

Just use epoch or ISO1601.

You're preaching to the choir here. Hell is other people's code.

Re: Parsing JSON is a Minefield

#98
post #56
post #19

Earlier quoted context omitted.

> (b) it's an OK choice though I really think it's not an OK choice. A config file format that doesn't allow comments provides some of the worst possible UX. One of the nice things about config files is that normally they are self-documenting, explaining the meaning of the various directives and providing possible values. Without comments, you have to constantly switch between the documentation and the config file. A…

How does json "not support comments"? {"comment":"default values for this object"}

For one thing, not every place where you might want a comment happens to be in an object.

Re: Parsing JSON is a Minefield

#99
post #32
post #14

Well, first and most obviously, if you are thinking of rolling your own JSON parser, stop and seek medical attention. Secondly, assume that parsing your input will crash, so catch the error and have your application fail gracefully. This is the number one security issue I encounter in "security audited" PHP. (The second being the "==" vs. "===" debacle that is PHP comparison.) As one example, consider what happens wh…

A parser should never crash on bad input. If it does, that's a serious bug that needs immediate attention, since that's at least a DoS vulnerability and quite likely something that could result in remote code execution. You definitely need to assume that the parser could fail , but that's different. Unless you're using "crash" in some way I'm not familiar with?

Some parsers are dealing with known good input, and should not include validation code for performance reasons. Often you will parse the same JSON many times throughout a pipeline, but you only really need to validate it once. A good example of this is a scatter-gather message bus, where the router parses a message record, scatters it to a large number of peers, and gathers responses. Depending on how latency-critical this system is, it can make sense to validate and normalize the representation at the router, and send normalized (whitespace-free) JSON to the compute nodes.

Re: Parsing JSON is a Minefield

#100

There was a great article at some point that explained why 'be liberal in what you accept' is a very bad engineering practice in certain circumstances, such as setting a standard, because it causes users to be confused and annoyed when a value accepted by system A is subsequently not accepted by supposedly compatible system B. Leading to pointless discussions about what the spec 'intended' and subtle incompatibility.…

Perhaps "The Harmful Consequences of Postel's Maxim"?:

https://news.ycombinator.com/item?id=9824638

If not, perhaps one of these:

http://programmingisterrible.com/post/42215715657/postels-pr...

https://bitworking.org/news/There_are_no_exceptions_to_Poste...

http://trevorjim.com/postels-law-is-not-for-you/

Post reply on HN