Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parsing JSON is a Minefield

seriot.ch

Post reply on HN
Print view View original

251–260 of 301 posts

Re: Parsing JSON is a Minefield

#251

Post by int_19h »

mypalmike wrote: post #222
clock_tower wrote: post #195

Earlier quoted context omitted.

Throwing an exception when parsing fails, sounds like a case of exception-handling as flow of control: a bad thing even when commonly done, having a lot in common with GOTO statements. (See http://softwareengineering.stackexchange.com/questions/18922... , and Ward's Wiki when it comes back up.)

Exceptions are for flow control. That's their entire purpose. Throwing an exception when parsing fails is a near perfect example of where exceptions produce clarity of code. That is to say, a parse function is usually a pure function that takes a string and returns some sort of object. As a pure function, it "answers a question". A parser answers the question: "What object does this string represent?" When given bad…

To be more specific (and your parser example actually makes it very clear), exceptions are a form of an error monad. The benefit that you describe - the ability to automatically flow throw error values without having to check for them at every boundary - is exactly what a monad provides.

The problem with exceptions is that they're not type-checked in most languages - you can throw whatever you want, and the type system doesn't reflect that, so there's no way to statically determine that all expected exceptions are properly handled.

They're (partially) type-checked in Java, but it's extremely annoying, because there's no ability to express that part of the type in a way that lets you write methods that are generic with respect to exceptions thrown by other code (e.g. there's no way to describe a method `f(T x)` such that it throws `E` plus everything that `x.g()` may throw, for any `T`).

Re: Parsing JSON is a Minefield

#252

Post by int_19h »

inimino wrote: post #224
jacquesm wrote: post #117

Earlier quoted context omitted.

> It is pretty rare to need to parse JSON yourself (what environment doesn't have that available?) but it isn't that difficult. It's a simple language. That, coupled with the fact that it is still so easy to get it wrong and to introduce security issues is exactly what should peak your attention to the seriousness of the subject. Building any parser is fraught with risk, it is super easy to get it subtly and horribly…

Writing any code is fraught with risk, but writing a parser in a modern and reasonably safe language is not something to be greatly feared. It's more likely that you'll introduce a security issue in what you do with the JSON immediately after you parse it.

> writing a parser in a modern and reasonably safe language is not something to be greatly feared

It ought to be feared, if interoperability is involved. The problem isn't that you might introduce security issues. The problem is usually that you introduce very subtle deviations to the spec that everyone else implemented correctly, and as a result, sometimes your input and/or output do not work with other stuff out there.

Re: Parsing JSON is a Minefield

#253

Post by amake »

grondo4 wrote: post #188
oblio wrote: post #145

Earlier quoted context omitted.

If your language doesn't come with JSON in the stdlib, you're really on the cutting edge. Or using a language meant for embedding :)

Doesn't Java lack a JSON parser in the standard library?

As of Java 8, the Nashorn JavaScript engine is included by default, and it does support JSON parsing.

[1] http://winterbe.com/posts/2014/04/05/java8-nashorn-tutorial/

Re: Parsing JSON is a Minefield

#254

Post by int_19h »

oblio wrote: post #145
ben_jones wrote: post #141

Earlier quoted context omitted.

Silent moment for those of us using niche languages to meet production requirements in environments that do not allow third-party code and do not have JSON parsing in the std lib...

If your language doesn't come with JSON in the stdlib, you're really on the cutting edge. Or using a language meant for embedding :)

> you're really on the cutting edge.

Or the other way around. Think about something like MUMPS.

Re: Parsing JSON is a Minefield

#255

Post by int_19h »

seagreen wrote: post #201
atombender wrote: post #196

Earlier quoted context omitted.

The spec is too vague; it's "agnostic". A number can include a fractional part, but it doesn't say if that means that fractionless numbers are to be treated as integers. It leaves that to the implementation, which is a terrible idea for an interchange format. Ruby, IMHO, errs on the side of consistency (fractionless numbers become Fixnum, so encode(decode("1")) => "1"), whereas Go goes the opposite way (all numbers b…

Thanks for your analysis (and the Ruby and JS reports). The more I think about it the more I think parsers should not parse to numbers by default, but instead to (using a Haskell-based pseudo type): data Sign = Positive | Negative data Digit = Zero | One ... Eight | Nine data JSONNumber = JSONNumber { _sign :: Maybe Sign , _integerPart :: [Digit] , _fractionalPart :: Maybe [Digit] } Actually this should include expon…

It sounds like what you're saying is that it should be deserialized as Java's BigDecimal, or whatever the equivalent of that is in a given/language framework; and if there isn't one, then the JSON parser should provide that equivalent.

Re: Parsing JSON is a Minefield

#256

Post by ubernostrum »

marxidad wrote: post #85
herge wrote: post #35

Earlier quoted context omitted.

What about raise an exception?

Exceptions should only be used for exceptional cases. For a parser, bad input should be expected.

You must really dislike Python, then since exceptions are used for all sorts of things that aren't unusual.

Re: Parsing JSON is a Minefield

#257

Post by ubernostrum »

userbinator wrote: post #250
Animats wrote: post #124

Earlier quoted context omitted.

JSON at least has the concept of "invalid JSON". That's a big step forward. A JSON parser, like an XML parser, can say "Syntax error - rejected." There's no such thing as "invalid HTML". For that reason, parsing HTML is a huge pain. As someone who has a web crawler, I'm painfully aware of how much syntactically incorrect HTML is out there. HTML5 has a whole section which standardizes how to parse bad HTML. That's jus…

There's no such thing as "invalid HTML". For that reason, parsing HTML is a huge pain. Actually, as someone who has written an HTML parser by following the HTML5 spec, I see it as the opposite: because every string of bytes essentially corresponds to some HTML tree, there are no special "invalid" edge cases to consider and everything is fully specified. That's the best situation, since bugs tend to arise at the edge…

I've written implementations of the HTML5 color algorithms. There are some sequences of bytes which, when given as a color value in HTML5, don't correspond to an RGB color, which makes things interesting.

(for the record, they are the empty string, and any string that is an ASCII case-insensitive match for the string "transparent")

Re: Parsing JSON is a Minefield

#258

Post by Whitestrake »

Volt wrote: post #248
userbinator wrote: post #245

Earlier quoted context omitted.

From experience, I think the whole "human-readable" idea is a bit overrated. All it means is that the format is entirely/mostly in ASCII. But if you have a hex editor, like all good programmers should, binary formats are not any less human-readable (or writable) nor more difficult to work with; and for some, even a text editor with CP437 or some other distinctive SBCS will suffice after a while. It's somewhat like le…

But the phrase is "human-readable" and not "programmer-readable".

A minor gripe with your comment, but as a programmer conceivably must be human, both conditions are satisfied when a programmer is capable of reading it.

Re: Parsing JSON is a Minefield

#259

Post by joesb »

TazeTSchnitzel wrote: post #164
_wiv7 wrote: post #10

"NaN and Infinity" Yeah. And I learned this the hard way with the Perl module JSON::XS. It successfully encodes a Perl NaN, but its decoder will choke on that JSON. (Reported it to the maintainer who insists that is consistent with the documentation and wouldn't fix it)

Similarly, Python's encoder violates the JSON specification by default, as it produces `Infinity`, `NaN` and `-NaN`, which other JSON parsers choke on.

I don't get it. Why would unaware JSON parsers choke on `Infinitiy`, `NaN` or `-Nan`? JSON has no concept of schema.

So if a parser sees "Inifinity", which it doesn't have any concept of, why would it do anything except treating that as string of a word "Infinity"?

Re: Parsing JSON is a Minefield

#260

Post by cdmckay »

joesb wrote: post #259
TazeTSchnitzel wrote: post #164

Earlier quoted context omitted.

Similarly, Python's encoder violates the JSON specification by default, as it produces `Infinity`, `NaN` and `-NaN`, which other JSON parsers choke on.

I don't get it. Why would unaware JSON parsers choke on `Infinitiy`, `NaN` or `-Nan`? JSON has no concept of schema. So if a parser sees "Inifinity", which it doesn't have any concept of, why would it do anything except treating that as string of a word "Infinity"?

Because it's not quoted like a string, it's a literal
Post reply on HN