Crap like this is why people should just use older ones that work. Some of the issues I see in the comments weren't present in Sun's XDR:
https://tools.ietf.org/html/rfc4506
Or even LISP s-expressions if you want organized text.
Parsing JSON is a Minefield
211–220 of 301 posts
Re: Parsing JSON is a Minefield
#212Well, first and most obviously, if you are thinking of rolling your own JSON parser, stop and seek medical attention. Secondly, assume that parsing your input will crash, so catch the error and have your application fail gracefully. This is the number one security issue I encounter in "security audited" PHP. (The second being the "==" vs. "===" debacle that is PHP comparison.) As one example, consider what happens wh…
A parser should never crash on bad input. If it does, that's a serious bug that needs immediate attention, since that's at least a DoS vulnerability and quite likely something that could result in remote code execution. You definitely need to assume that the parser could fail , but that's different. Unless you're using "crash" in some way I'm not familiar with?
In "safe" languages (i.e. ones which aren't supposed to segfault/etc unless there's a compiler/interpreter/VM bug -- PHP, Java, JS, Python all fit in here) I've often heard "crash" being used to just mean an uncaught exception, or something else that brings the program crashing down.
GP says "or catch the error", so they're probably using that term in this sense.
Re: Parsing JSON is a Minefield
#213Earlier quoted context omitted.
JSON's number support is a source of problems, because the standard is so informal [1]: JSON is agnostic about numbers. ... JSON instead offers only the representation of numbers that humans use: a sequence of digits This poses a problem for some languages, and tends to break things. You would expect encode(decode(string)) == string, but languages deal with numbers differently. For example, in Go, if you decode into…
Exactly. JSON Number ARE NOT ACTUAL NUMBERS. They're really restricted strings (or, as you quote, a syntax for representing numbers). IMO this wasn't originally a bad thing at all. There are so many different types of numbers, with so many different behaviours (does `1` == `1.0`? Not in statistics class) that trying to work it all out in JSON would have been a fools errand. The problem is that so many JSON parsers ar…
> JSON Number ARE NOT ACTUAL NUMBERS.
Well, no. They're numbers as humans think about them.
(Cue the saying about programming being a job where you hate yourself for not thinking enough like a computer)
Re: Parsing JSON is a Minefield
#214Earlier quoted context omitted.
Throwing an exception when parsing fails, sounds like a case of exception-handling as flow of control: a bad thing even when commonly done, having a lot in common with GOTO statements. (See http://softwareengineering.stackexchange.com/questions/18922... , and Ward's Wiki when it comes back up.)
But exception handling is flow control, by its very nature. So it's clearly a gray area and the right thing to do depends on the common idioms of the language you're using, the expected frequency of parsing failures, and (possibly) runtime performance concerns. In Java for example, the XML parser built into the standard library does throw exceptions for certain types of invalid input.
But see Djikstra on "GOTO statement considered harmful" (http://david.tribble.com/text/goto.html). The problem is unstructured control flow, which both GOTOs and exceptions-as-control-flow give you; at least in what I was taught (early-2000s CS degree focused on C++), unstructured flow of control is only acceptable when it's a panic button to quit the program (or a major area of processing).
It sounds like the Web way of doing things doesn't have this tradition -- much like how it doesn't have static strict extensible type systems.
Re: Parsing JSON is a Minefield
#215The correct answer to parsing JSON is... don't. We experimented last hackday with building Netflix on TVs without using JSON serialization (Netflix is very heavy on JSON payloads) by packing the bytes by hand to get a sense of how much the "easy to read" abstraction was costing us, and the results were staggering. On low end hardware, performance was visibly better, and data access was lightening fast. Michael Paulso…
You may want to take a look at Protocol Buffers or the new Ion format by Amazon; the latter can seamlessly switch between binary and human-readable.
Re: Parsing JSON is a Minefield
#216Earlier quoted context omitted.
YAML has braces. In fact, it's a super set of JSON. Any YAML parser should be able to parse JSON encoded data with one exception. Block comments. Which is a bastardization of JSON (as mentioned in the article) so block comments shouldn't be a problem in most cases.
> [YAML is] a super set of JSON. The specification says this, but I was never convinced it was true. Specifically, the spec around escaped unicode characters lacks any mention of surrogates being encoded in two \u sequences, but rather specifies \u as: > Escaped 16-bit Unicode character. Which is an unhelpfully not-even-wrong statement. In practice, trying to treat JSON as a subset of YAML results in things not round…
That's not wrong.
In [1]: import json
In [2]: json.dumps("\N{PILE OF POO}")
Out[2]: '"\\ud83d\\udca9"'
YAML specifies that the recommended encoding is UTF-8, but should be able to parse UTF-16 and UTF-32. And, if you want that, then you need to tell YAML to expect UTF-16. You can do that by including a BOM (Byte-Order Mark).
PyYAML is documented as only supporting UTF-8 and UTF-16, but not UTF-32.
I'm scratching my head on how to get that "UTF-16 encoded" UTF-8 string to a proper representation (in Python). If this were Ruby, I would use str.force_encoding() which doesn't change the bytes at all. If the solution comes to be, then I'll update this comment (or reply).
http://pyyaml.org/wiki/PyYAMLDocumentation http://yaml.org/spec/1.2/spec.html#id2771184
Re: Parsing JSON is a Minefield
#217Earlier quoted context omitted.
For one thing, not every place where you might want a comment happens to be in an object.
Like where? Who for? For what purpose? I cant think of a single example that this would be useful.
A list of items in an array. Some items in that list are of particular note.
(Say you have 15 items in an array. For whatever reason, such as you not being in control of the expected input structure of whatever you're giving this JSON to, you have them grouped with comments at the top of each block.)
Re: Parsing JSON is a Minefield
#218Earlier quoted context omitted.
Funnily enough, as I've been experimenting with Chef and trying to stick to JSON config files where allowed, I was again struck that (a) it's not a good choice for config files (b) it's an OK choice though (c) lots of people are using it anyway (d) nearly everyone that does so (including Chef) allows comments, so in reality are not actually using JSON at all. Point (d) is the important one. I really think we need a s…
> (b) it's an OK choice though I really think it's not an OK choice. A config file format that doesn't allow comments provides some of the worst possible UX. One of the nice things about config files is that normally they are self-documenting, explaining the meaning of the various directives and providing possible values. Without comments, you have to constantly switch between the documentation and the config file. A…
> I really think it's not an OK choice. A config file format that doesn't allow comments provides some of the worst possible UX.
Are we sure about that? E.g. mostly when messing with configuration files, I have to visit the documentation anyway, which could explain key-value pairs in the json file. Furthermore there are configuration files which consist mostly of comments to explain all kind of edge cases with actual configuration data commented, which can be a mess on its own.
If good code should be self-explanatory then why not configurations?
Re: Parsing JSON is a Minefield
#219Earlier quoted context omitted.
But exception handling is flow control, by its very nature. So it's clearly a gray area and the right thing to do depends on the common idioms of the language you're using, the expected frequency of parsing failures, and (possibly) runtime performance concerns. In Java for example, the XML parser built into the standard library does throw exceptions for certain types of invalid input.
But see Djikstra on "GOTO statement considered harmful" ( http://david.tribble.com/text/goto.html ). The problem is unstructured control flow, which both GOTOs and exceptions-as-control-flow give you; at least in what I was taught (early-2000s CS degree focused on C++), unstructured flow of control is only acceptable when it's a panic button to quit the program (or a major area of processing). It sounds like the Web…
Quoting Dijkstra from 1968 on goto as argumentum ad authority in 2016 should be considered harmful.
He had reasons for saying what he said in the context of the times. Remove that temporal context and the why becomes more important than the what.
Re: Parsing JSON is a Minefield
#220Earlier quoted context omitted.
The spec is too vague; it's "agnostic". A number can include a fractional part, but it doesn't say if that means that fractionless numbers are to be treated as integers. It leaves that to the implementation, which is a terrible idea for an interchange format. Ruby, IMHO, errs on the side of consistency (fractionless numbers become Fixnum, so encode(decode("1")) => "1"), whereas Go goes the opposite way (all numbers b…
Thanks for your analysis (and the Ruby and JS reports). The more I think about it the more I think parsers should not parse to numbers by default, but instead to (using a Haskell-based pseudo type): data Sign = Positive | Negative data Digit = Zero | One ... Eight | Nine data JSONNumber = JSONNumber { _sign :: Maybe Sign , _integerPart :: [Digit] , _fractionalPart :: Maybe [Digit] } Actually this should include expon…
> At the moment I don't even feel like I can write a protocol that expected implementations to distinguish `1` and `1.0`.
Indeed you can't, among other reasons because that distinction doesn't exist in JavaScript, but ... why would you want to make such a subtle distinction?
If you really want what you described above, you can get it by using a string.