Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unintuitive JSON Parsing

nullprogram.com

Post reply on HN
Print view View original

51–60 of 77 posts

Re: Unintuitive JSON Parsing

#51

Post by userbinator »

magicalhippo wrote: post #46
ChrisSD wrote: post #30

Earlier quoted context omitted.

JSON was made to be "based on a subset" of Javascript. The only way to be compatible with JS while removing octals is to disallow leading zeroes entirely. Doing otherwise would lead to JSON and JS behaving differently with the same input. Of course, until recently JSON wasn't a strict subset of JS but that was an oversight rather than by design.

I've been programming for 30 years, across many different languages from assembler and up. I've yet to use octals for any code. What am I missing out on? Why are they included in modern languages like JS?

Almost certainly due to C heritage.

Re: Unintuitive JSON Parsing

#52

Post by esfandia »

ufo wrote: post #18
repsilat wrote: post #9

Earlier quoted context omitted.

This seems more flexible to me somehow. Using the approach in TFA, can the laser handle tokens that are prefixes of other tokens? Or even tokens that share prefixes? Lexing "truefalse" as two adjacent tokens "true" and "false" seems slightly crazier than just lexing it as one (meaningless) token.

Usually, if there is a token that is a prefix of another the longer token wins. For example, in Javascript --x is parsed as the decrement operator instead of as two unary "-" operators.

With that rule, the "leading zero" problem in the article should have been caught properly as well. Why wasn't it?

edit: I guess because "01" isn't a valid token.

Re: Unintuitive JSON Parsing

#53

Post by zAy0LfpBZLC8mAC »

donpdonp wrote: post #23

Its possible the parser was laid out to choke on octals as a way to protect the 'standard'. Its one decision to not support octals, and its another to make octal-style an error so those numbers are not parsed as base10.

You have it all backwards. What a string in any language means is defined by the language specification. If the JSON spec doesn't say that '01' is an octal number, then it's not an octal number. What you would like it to be, or what other language specs say, is completely irrelevant for what that string means in JSON.

Also, making a language X parser accept anything that is not in fact language X is nothing but a terrible idea. If there is one thing that standards are good for, it's interoperability. And if there is one thing that hurts interoperability, it's having different implementations of supposedly the same standard accept and reject different inputs. That's how you get websites that work in one browser, but not another, because one browser was so helpful to make up some meaning for your creative markup instead of rejecting it with an error message, which obviously helps you absolutely nothing with the next browser that is of a different opinion. If you think the spec is stupid, you have to change the spec, if you don't manage to do that, you still should implement the spec, because interoperability is more important than whether your program can read some input that isn't JSON and that therefore no other JSON parser is guaranteed to understand anyway.

Re: Unintuitive JSON Parsing

#54

Post by will4274 »

PeterisP wrote: post #20
will4274 wrote: post #11

Why is concatenated json a thing? In what sense is: {0}{1} better than [{0},{1}]? Presumably, if a few bytes are a major concern, you aren't using JSON anyway.

If you have a file or network stream with millions of separate JSON items, then you might want to parse and process each item separately as it is received, and the surrounding structure just gets in the way. That being said, it's properly better to explicitly acknowledge that you're using something-like-json-but-not-really-json like http://jsonlines.org does instead of simply concatenating json objects.

With the application understanding that the top level object is an array of independently parsable json objects, it should still be possible to stream the format I suggested, assuming you use a streaming / sax parser.

Re: Unintuitive JSON Parsing

#55

Post by setr »

svnpenn wrote: post #44
ekimekim wrote: post #42

Earlier quoted context omitted.

Octal notation is traditionally used in several contexts - file mode probably being the most common. If you were writing a JSON object to describe a file to be created, and you were under the mistaken impression that JSON supported octal with a leading zero (like most languages), it would be entirely reasonable to write something like: { "path": "/foo", "mode": 0644, "contents": "bar" }

Yeah ok, but its also explicitly not allowed by the specification, both in text: > A number is very much like a C or Java number, except that the octal and hexadecimal formats are not used. and image: https://json.org/img/number.png as shown literally on the JSON home page: https://json.org I am all for good error handling, but at some point you do have to blame the user.

Error messages are "blaming the user".. its job is to help inform the user the mistake he made.

You can silently beat your child everytime he makes a mistake, until he accidentally does the job correctly (and doesn't get beaten), but it seems to me that making use of our ability to communicate can be much more efficient (and significantly less painful for the child).

And json is merely a (very innefficient, and somewhat problematic) protocol for information exchange; it's not something you should expect people to have read the spec for, especially when its whole popularity stems from it being "intuitive" -- that is, you don't really need to read the spec to deal with it effectively

Re: Unintuitive JSON Parsing

#56

Post by johnhenry »

reificator wrote: post #31
johnhenry wrote: post #27

> Either the leading zero is ignored, or it indicates octal, as it does in many languages, including JavaScript. This is false. In JavaScript, a leading zero, unless accompanied by a lowercase oh ('o') does indicate the number is written in octal. 08 === 0o10; // true Here, the left side is still base 10, while the right side is base 8.

I wouldn't say it's false, I would say it's no longer true in strict mode . https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... Javascript for many years has assumed a leading zero means an octal prefix, and it's only recently that behavior has changed. EDIT: Also your example does not disprove this. Here's another example that you should try running in the console: 011 === 11; // false 011; // 9

Thanks for the link!

EDIT: Also, yes, I the mistake with my initial example -- as '8' doesn't exist in octal, we have no choice but to interpret the leading '0' as padding and '08' as base ten, whereas '11' can be interpreted as base eight.

Re: Unintuitive JSON Parsing

#57

Post by Jasper_ »

jchook wrote: post #33
will4274 wrote: post #13

Earlier quoted context omitted.

Not sure what's confusing. "[01]" is not valid JSON. JSON being a subset of JavaScript means that all valid JSON constructs are valid JavaScript constructs. So, the subset statement says nothing at all about "[01]"

That said, in theory, shouldn't: JSON.parse("0o10") === 8? I get SyntaxError: Unexpected token o in JSON at position 1

JSON.parse("(function() { })") doesn't parse either, despite being valid JavaScript. Both constructs are specific to JavaScript, and do not exist in JSON. Hence, JSON is a subset of JavaScript.

Re: Unintuitive JSON Parsing

#58

Post by ComputerGuru »

jchook wrote: post #25
ComputerGuru wrote: post #15

So is it incorrect (Technically, at any rate) for a parser to support leading zeroes in its implementation?

Yep. It's technically incorrect. It seems honoring this type of technical correctness matters a lot. For example, imagine if ECMA added a new feature (e.g. 0-prefixed octal literals) in 2020.. Another issue: security. Imagine a hacker figured out that you used a mix of JSON parsers on your application (e.g. V8 and jq), and they produced different output. For a vaguely related example, consider that some URL parsers i…

I believe what you're actually saying is that regardless of whether or not it is technically correct, it would be incorrect (and I agree with you there).

My question was more "for inputs not defined as being valid by the spec, is the result undefined (a la C++ UB where anything and everything is legal in response) or is it required to reject said input".

The sibling response says extensions are allowed, but that wouldn't come into play if an input is specifically called out as disallowed (vs simply not taken into account whatsoever).

Re: Unintuitive JSON Parsing

#59

Post by [deleted] »

faho wrote: post #35
ComputerGuru wrote: post #15

So is it incorrect (Technically, at any rate) for a parser to support leading zeroes in its implementation?

RFC8259 allows arbitrary extensions, so as long as it doesn't cause any misparses of regular JSON it would be acceptable, at least up to that standard.

[deleted]

Re: Unintuitive JSON Parsing

#60

Post by [deleted] »

ComputerGuru wrote: post #58
jchook wrote: post #25

Earlier quoted context omitted.

Yep. It's technically incorrect. It seems honoring this type of technical correctness matters a lot. For example, imagine if ECMA added a new feature (e.g. 0-prefixed octal literals) in 2020.. Another issue: security. Imagine a hacker figured out that you used a mix of JSON parsers on your application (e.g. V8 and jq), and they produced different output. For a vaguely related example, consider that some URL parsers i…

I believe what you're actually saying is that regardless of whether or not it is technically correct, it would be incorrect (and I agree with you there). My question was more "for inputs not defined as being valid by the spec, is the result undefined (a la C++ UB where anything and everything is legal in response) or is it required to reject said input". The sibling response says extensions are allowed, but that woul…

[deleted]
Post reply on HN