Live data from Hacker News

On-demand JSON: A better way to parse documents?

onlinelibrary.wiley.com

21–30 of 55 posts

Re: On-demand JSON: A better way to parse documents?

#21
post #15

Sorry, I would never use this. Before I consume any json from any source or for any purpose I validate it. Lazy loading serves no purpose if you need validation. Hint: you need validation.

There are cases where the json does not come from a user input and can be trusted without a validation layer. Also you may want to stream-validate it.

[dead]

Re: On-demand JSON: A better way to parse documents?

#23
post #10

> The JSON syntax is nearly a strict subset of the popular programming language JavaScript. What JSON isn’t valid JS?

"Any JSON text is a valid JavaScript expression, but only after the JSON superset revision. Before the revision, U+2028 LINE SEPARATOR and U+2029 PARAGRAPH SEPARATOR are allowed in string literals and property keys in JSON; but the same use in JavaScript string literals is a SyntaxError." https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... "In fact, since JavaScript does not support bare objects, the simp…

Eh, this is slightly dated—{"k":"v"} does work in the WebKit and Blink consoles, and the superset proposal was approved, so those separators should work fine too.

Re: On-demand JSON: A better way to parse documents?

#24
post #10

> The JSON syntax is nearly a strict subset of the popular programming language JavaScript. What JSON isn’t valid JS?

"Any JSON text is a valid JavaScript expression, but only after the JSON superset revision. Before the revision, U+2028 LINE SEPARATOR and U+2029 PARAGRAPH SEPARATOR are allowed in string literals and property keys in JSON; but the same use in JavaScript string literals is a SyntaxError." https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... "In fact, since JavaScript does not support bare objects, the simp…

> "In fact, since JavaScript does not support bare objects, the simple statement {"k":"v"} will emit an error in JavaScript"

This is kind of a silly, "well technically". Its a valid expression. Its not a valid statement. It is valid javascript in the sense most people mean when asking the question if something is valid javascript.

Re: On-demand JSON: A better way to parse documents?

#25
> The JSON specification has six structural characters (‘[’, ‘{’, ‘]’, ‘}’, ‘:’, ‘,’) to delimit the location and structure of objects and arrays.

Wouldn’t a quote “ also be a structural character? It doesn’t actually represent data, it just delimits the beginning and end of a string.

I get why I’m probably wrong: a string isn’t a structure of chars because that’s not a type in json. The above six are the pieces of the two collections in JSON.

Re: On-demand JSON: A better way to parse documents?

#26

Why not just use msgpack? The advantage of JSON is that support is already built in to everything and you don't have to think about it. If you start having to actually make an effort to fuss with it, then why not consider other formats? This does have nice backwards compatibility with existing JSON stuff though, and sticking to standards is cool. But msgpack is also pretty nice.

This seems to be geared towards using a heavily adopted format.

Some would want to move to binary, but it's hard to find an ideal universal binary format.

msgpack doesn't support a binary chunk bigger than 4gb, which is unfortunate. Also the JavaScript library doesn't handle Map vs plain object.

In JSON you could have a 10GB Base64 blob, such as a video, in a string, no problem (from the format side, with a library YMMV).

For one that supports up 64 bit lengths, check out CBOR: https://cbor.io/ With libraries maybe it could be the ideal universal binary format (universal in the same sense JSON is - I've heard it called that). https://www.infoworld.com/article/3222851/what-is-json-a-bet...

Re: On-demand JSON: A better way to parse documents?

#28
post #13

Sorry, I would never use this. Before I consume any json from any source or for any purpose I validate it. Lazy loading serves no purpose if you need validation. Hint: you need validation.

You need a parser for validation, - preferably a fast, possibly even a streaming one.

Which this is not.

A validating parser, that is. The paper clearly indicates that invalid JSON like [1, 1b] will pass, unless your code happens to try and decode the 1b.

Re: On-demand JSON: A better way to parse documents?

#29

Why not just use msgpack? The advantage of JSON is that support is already built in to everything and you don't have to think about it. If you start having to actually make an effort to fuss with it, then why not consider other formats? This does have nice backwards compatibility with existing JSON stuff though, and sticking to standards is cool. But msgpack is also pretty nice.

Example: you work on the mobile team, the backend team is large and focuses on serving the web app, they send huge JSON payloads that the mobile app only partially need, and asking the backend team to now also serve msgpack is out of the question as things together with the backend and web teams were proven to be a PITA.

In this scenario, writing a new, or bundling someone else's json library can significantly improve things.

Re: On-demand JSON: A better way to parse documents?

#30
This is a real “why didn’t I think of that” moment for sure. So many systems I’ve written have profiled with most of the cpu and allocations in the JSON parser, when all it needs is a few fields. But rewriting it all in SAX is just not worth all the trouble.
Post reply on HN