Unintuitive JSON Parsing
nullprogram.com
Unintuitive JSON Parsing
1–10 of 77 posts
Re: Unintuitive JSON Parsing
#2> The parser will not complain about leading zeros because JSON has no concept of leading zeros.
Of course there is no logical reason why the parser shouldn't have this concept just because the spec doesn't require. IMO, beyond basic correctness, user friendly error messages are the main differentiator between excellent parsers and crappy parsers.
Re: Unintuitive JSON Parsing
#3The article leads with an incorrect statement. JSON is now a subset of JavaScript: https://github.com/tc39/proposal-json-superset
Re: Unintuitive JSON Parsing
#4> The parser will not complain about leading zeros because JSON has no concept of leading zeros. Of course there is no logical reason why the parser shouldn't have this concept just because the spec doesn't require. IMO, beyond basic correctness, user friendly error messages are the main differentiator between excellent parsers and crappy parsers.
Reporting the last valid input and the start of the first invalid location (possibly repeating the first couple characters of invalid content, filtered for safety) is what I'd generally prefer in an error message.
Re: Unintuitive JSON Parsing
#5In cases like this, the parser and lexer can often produce a better error message if they are written to accept a more lax input and then check it for errors.
For example, instead of faithfully implementing the grammar from the specification, allow numbers with leading zeroes and then produce an error for them.
Another situation where this comes up is parsing language keywords. Instead of writing a separate lexer rule for every keyword, write a single rule for keyword-or-identifier, and then use a hash table lookup inside of that to determine if it is a keyword or identifier.
Re: Unintuitive JSON Parsing
#6> The parser will not complain about leading zeros because JSON has no concept of leading zeros. Of course there is no logical reason why the parser shouldn't have this concept just because the spec doesn't require. IMO, beyond basic correctness, user friendly error messages are the main differentiator between excellent parsers and crappy parsers.
Browser JSON parsers probably shouldn't be optimized for user friendliness, since to the users running the program any error message is going to be unfriendly. They are optimized for speed, and if nice error messages slow thing down at all, then bare messages it is.
Maybe there's a way to re-parse with a friendly parser with there's an error and dev tools is open.
Re: Unintuitive JSON Parsing
#7I was initially surprised that all the lexers treat "[01]" as four tokens, but it makes sense from the state diagram.
In the past I've encountered JSON lexing that only considers token boundaries on "special" characters i.e. ",}]:" and whitespace. This will return a lexing error when it sees "01" (equivalently "truefalse").
Re: Unintuitive JSON Parsing
#8The article leads with an incorrect statement. JSON is now a subset of JavaScript: https://github.com/tc39/proposal-json-superset
According to the proposal, this has been shipped in V8 in Chrome 66 (from the V8 bug report: The ECMAScript ⊃ JSON proposal shipped in V8 v6.6 and Chrome 66). And yet my Chrome version 79 does not parse "[01]", throws the same error as described in the article. Same error in Node 12.14.0 (which includes V8 7.7.299.13). Something doesn't add up.
Re: Unintuitive JSON Parsing
#9In cases like this, the parser and lexer can often produce a better error message if they are written to accept a more lax input and then check it for errors. For example, instead of faithfully implementing the grammar from the specification, allow numbers with leading zeroes and then produce an error for them. Another situation where this comes up is parsing language keywords. Instead of writing a separate lexer rul…
This seems more flexible to me somehow.
Using the approach in TFA, can the laser handle tokens that are prefixes of other tokens? Or even tokens that share prefixes?
Lexing "truefalse" as two adjacent tokens "true" and "false" seems slightly crazier than just lexing it as one (meaningless) token.
Re: Unintuitive JSON Parsing
#10> The parser will not complain about leading zeros because JSON has no concept of leading zeros. Of course there is no logical reason why the parser shouldn't have this concept just because the spec doesn't require. IMO, beyond basic correctness, user friendly error messages are the main differentiator between excellent parsers and crappy parsers.
I agree. Often, it's much easier to generate good diagnostic messages by adding failure paths to lexer or parser definitions. i.e. in the 01 case, add a rule for numbers with leading zeroes to the lexer, which generates an error explaining that leading zeroes are prohibited.