another example of using "minefield" when you mean "has more than 10 rules"
Parsing JSON is a Minefield
211–220 of 257 posts
Re: Parsing JSON is a Minefield
#212While this is true of JSON, it's also true of any other non-trivial serialization and/or encoding format. The main lessons to learn here are that: 1) implementation matters 2) "simple" specs never really are It's definitely important to have documents like this one that explore the edge cases and the differences between implementations, but you can replace "JSON" in the introductory paragraph with any other serializa…
Paring "csv" and "obj" format is also difficult. They both have "simple" specs, but neither of them has a standard spec.
They both have after-the fact specifications that (try to) codify various flavours that had already been released into the wild. (I assume you mean Wavefront Obj). It's a pity that they aren't better specified. The same holds for ini. Maybe that's just the curse of good engineering: If it's simple, it's easy to make a mess out of it.
Re: Parsing JSON is a Minefield
#213Earlier quoted context omitted.
It really isn't true for JSON either. If you read the rant, most of it is simply about JSON parsers accepting additional, non-JSON syntaxes. Looking at the matrix, all green, yellow, light blue and dark blue are OK outcomes. Red are crashes (stack overflow with 10000 nested arrays, for example), and dark brown are valid JSON that didn't parse (things like UTF-8 mishandling). The issues aren't really JSON-specific.
So if JSON parsers can't agree on what JSON actually is, that isn't a problem?
JSON parsers all agree what JSON actually is (except for the brown colored fields in the compat matrix, which are actual JSON compat bugs).
JSON parsers may also permit various additional variations which are very explicitly not JSON. This means that they may accept a handcrafted, technically invalid JSON document. However, a JSON encoder may never generate a document containing such "extensions", as this would not be JSON.
This concept of having parsers accept more than necessary follows best practices of robustness. As the robustness principle goes: "Be conservative in what you do, be liberal in what you accept from others".
Re: Parsing JSON is a Minefield
#214Try RSS. Having built a system years ago, to try an parse tens of thousands of feeds, there's a huge amount of 'fuzzy logic' required to put it all in order. Despite the spec.
There were nine different and incompatible versions.
http://web.archive.org/web/2004/http://diveintomark.org/arch...
Re: Parsing JSON is a Minefield
#215Earlier quoted context omitted.
Yeah, that spectrum is a good way to think about it. JSON hits a sweet spot of being very easy for computers to deal with almost all the time, while also being reasonably easy for humans to read and write. I was going to add “if you started with that as the spec, it wouldn’t be hard to design something better than JSON” but real examples like YAML are pretty awkward, so probably it’s a harder problem than it seems.
I think it's the other way round: JSON mimics the syntax of JavaScript literals and the syntax was meant to be easy for humans to write and read. Not super-easy, because it has to be easily parseable as well, but still the original use case of this notation is to write relatively short pieces of code. This is why it seems easy to people and they assume it's easy for computers as well, while in fact JSON is hard for c…
Yes, I really wish trailing commas were standard.
But they’re not, so we have to resist the temptation to use them!
Re: Parsing JSON is a Minefield
#216Earlier quoted context omitted.
JSON is fairly trivial. The post is a nonsensical rant about parsers accepting non-JSON compliant documents (as the JSON spec specifically states that parsers may), such as trailing commas. In the large colored matrix, the following colors mean everything is fine: Green, yellow, light blue and deep blue. Red are crashes (things like 10000 nested arrays causing a stack overflow—this is a non-JSON-specific parser bug),…
As a question of fact, programs put out JSON that gets misparsed by other programs. Some simply parse floating point values differently, or they treat Unicode strings incorrectly, or output them incorrectly. Different parsers have different opinions about what a document represents. This has a real world impact. Accepting invalid or ambiguous or undefined JSON is not an acceptable behavior. It means bugs get swallowe…
Unicode and floating point misparsing are not even remotely JSON related, but are simply bugs that can occur in any parser that handles unicode or floating point. Thus, complaining about it in a "JSON is a minefield" thread is a bit silly.
If you put out JSON that gets misparsed, you either generated invalid JSON, or the parser is faulty. Nothing around that.
This has nothing to do with whether parsers have flexibility to accept additional constructs, which is extremely common for a parser to do.
Re: Parsing JSON is a Minefield
#217Earlier quoted context omitted.
JSON is fairly trivial. The post is a nonsensical rant about parsers accepting non-JSON compliant documents (as the JSON spec specifically states that parsers may), such as trailing commas. In the large colored matrix, the following colors mean everything is fine: Green, yellow, light blue and deep blue. Red are crashes (things like 10000 nested arrays causing a stack overflow—this is a non-JSON-specific parser bug),…
Actually unless one is doing JavaScript, JSON is extremely difficult to parse correctly. I challenge you to write a simple, understandable JSON parser in Bourne shell or in AWK.
Why in the world would I write a parser in bash or awk, regardless of the format? I certainly have better things to do. It doesn't matter how sensible a format is, those tools are simply not appropriate to write a parser in.
I have, however, written a JSON parser before at a previous company in C++ (they didn't want to pull in a library). It wasn't particularly hard.
And yes, like any other parser, it accepted additional non-JSON constructs. This was simply because it was take additional work to error out on those constructs, which would be a waste of time.
Re: Parsing JSON is a Minefield
#218Earlier quoted context omitted.
Actually unless one is doing JavaScript, JSON is extremely difficult to parse correctly. I challenge you to write a simple, understandable JSON parser in Bourne shell or in AWK.
> JSON is extremely difficult to parse correctly ... in Bourne shell or in AWK. Sorry for the misquote, but does it get to the heart of your objection? I'm torn here. On the one hand I want to say "Those are not languages one typically writes parsers in," but that's a really muddled argument: 1. People "parse" things often in bash/awk because they have to -- because bash etc deal in unstructured data. 2. Maybe "reaso…
You definitely shouldn't write a parser for anything in bash.
If you want to deal with JSON, I'd recommend jq as an easy manipulation tool.
Re: Parsing JSON is a Minefield
#219Earlier quoted context omitted.
> JSON is extremely difficult to parse correctly ... in Bourne shell or in AWK. Sorry for the misquote, but does it get to the heart of your objection? I'm torn here. On the one hand I want to say "Those are not languages one typically writes parsers in," but that's a really muddled argument: 1. People "parse" things often in bash/awk because they have to -- because bash etc deal in unstructured data. 2. Maybe "reaso…
You definitely shouldn't write a parser for anything in bash. If you want to deal with JSON, I'd recommend jq as an easy manipulation tool.
I wrote Bourne shell and both of you assumed bash. Horrid state of affairs in today’s IT. All hail Linux hegemony!
And AWK, if it’s not easily parsable with the language specifically designed for parsing data, something is wrong with that data.
Re: Parsing JSON is a Minefield
#220Earlier quoted context omitted.
Actually unless one is doing JavaScript, JSON is extremely difficult to parse correctly. I challenge you to write a simple, understandable JSON parser in Bourne shell or in AWK.
> JSON is extremely difficult to parse correctly ... in Bourne shell or in AWK. Sorry for the misquote, but does it get to the heart of your objection? I'm torn here. On the one hand I want to say "Those are not languages one typically writes parsers in," but that's a really muddled argument: 1. People "parse" things often in bash/awk because they have to -- because bash etc deal in unstructured data. 2. Maybe "reaso…
Shell and AWK programs have fewest dependencies, are extremely light on resources’ consumption and are extremely fast. When someone’s program emits JSON because the author assumed that programs which ingest that data will also be JavaScript programs, that’s a really bad assumption. It would force the consumer of that data to replicate the environment. This goes against core UNIX principles, as discussed at length in “The art of UNIX programming” book. It’s a rookie mistake to make.