Parsing JSON is a Minefield

Post by seagreen » Wed, Oct 26, 2016, 9:49 PM UTC

atombender wrote: post #196

seagreen wrote: post #187

Earlier quoted context omitted.

Exactly. JSON Number ARE NOT ACTUAL NUMBERS. They're really restricted strings (or, as you quote, a syntax for representing numbers). IMO this wasn't originally a bad thing at all. There are so many different types of numbers, with so many different behaviours (does `1` == `1.0`? Not in statistics class) that trying to work it all out in JSON would have been a fools errand. The problem is that so many JSON parsers ar…

The spec is too vague; it's "agnostic". A number can include a fractional part, but it doesn't say if that means that fractionless numbers are to be treated as integers. It leaves that to the implementation, which is a terrible idea for an interchange format. Ruby, IMHO, errs on the side of consistency (fractionless numbers become Fixnum, so encode(decode("1")) => "1"), whereas Go goes the opposite way (all numbers b…

  data Sign = Positive | Negative

  data Digit = Zero | One ... Eight | Nine

  data JSONNumber = JSONNumber
    { _sign           :: Maybe Sign
    , _integerPart    :: [Digit]
    , _fractionalPart :: Maybe [Digit]
    }

Post by nradov » Wed, Oct 26, 2016, 10:03 PM UTC

clock_tower wrote: post #195

detaro wrote: post #106

Earlier quoted context omitted.

And in many languages, they are the common way to communicate that a function can not return the data that is expected of it. Invalid input data means the parser can't produce the equivalent data structure -> exception. If the parser has some kind of partial parsing, a way to recover from errors or you are using a language in which returning explicit errors is the more common idiom, then you probably shouldn't throw…

Throwing an exception when parsing fails, sounds like a case of exception-handling as flow of control: a bad thing even when commonly done, having a lot in common with GOTO statements. (See http://softwareengineering.stackexchange.com/questions/18922... , and Ward's Wiki when it comes back up.)

Post by nitrogen » Wed, Oct 26, 2016, 10:14 PM UTC

microcolonel wrote: post #99

mikeash wrote: post #32

Earlier quoted context omitted.

A parser should never crash on bad input. If it does, that's a serious bug that needs immediate attention, since that's at least a DoS vulnerability and quite likely something that could result in remote code execution. You definitely need to assume that the parser could fail , but that's different. Unless you're using "crash" in some way I'm not familiar with?

Some parsers are dealing with known good input, and should not include validation code for performance reasons. Often you will parse the same JSON many times throughout a pipeline, but you only really need to validate it once. A good example of this is a scatter-gather message bus, where the router parses a message record, scatters it to a large number of peers, and gathers responses. Depending on how latency-critica…

Post by twic » Wed, Oct 26, 2016, 10:24 PM UTC

Impossible wrote: post #131

realkitkat wrote: post #120

If JSON is comparable to minefield, then I guess XML and ASN.1 are nothing short of nuclear Armageddon in complexity and ones ability to shoot themselves into the leg ;-)

Also, as someone who has written an XML parser, according to some of the comments in this threads I'm way beyond medical help, and should give up on life :).

Post by dvt » Wed, Oct 26, 2016, 11:15 PM UTC

gcirino42 wrote: post #186

The correct answer to parsing JSON is... don't. We experimented last hackday with building Netflix on TVs without using JSON serialization (Netflix is very heavy on JSON payloads) by packing the bytes by hand to get a sense of how much the "easy to read" abstraction was costing us, and the results were staggering. On low end hardware, performance was visibly better, and data access was lightening fast. Michael Paulso…