Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

On-demand JSON: A better way to parse documents?

onlinelibrary.wiley.com

Post reply on HN
Print view View original

41–50 of 55 posts

Re: On-demand JSON: A better way to parse documents?

#41

Post by pitaj »

turnsout wrote: post #23
sp332 wrote: post #10

Earlier quoted context omitted.

"Any JSON text is a valid JavaScript expression, but only after the JSON superset revision. Before the revision, U+2028 LINE SEPARATOR and U+2029 PARAGRAPH SEPARATOR are allowed in string literals and property keys in JSON; but the same use in JavaScript string literals is a SyntaxError." https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe... "In fact, since JavaScript does not support bare objects, the simp…

Eh, this is slightly dated—{"k":"v"} does work in the WebKit and Blink consoles, and the superset proposal was approved, so those separators should work fine too.

The console evaluates what is passed to it as an expression, which is a different context from how scripts or modules are evaluated.

Re: On-demand JSON: A better way to parse documents?

#42

Post by secondcoming »

serial_dev wrote: post #29
eternityforest wrote: post #19

Why not just use msgpack? The advantage of JSON is that support is already built in to everything and you don't have to think about it. If you start having to actually make an effort to fuss with it, then why not consider other formats? This does have nice backwards compatibility with existing JSON stuff though, and sticking to standards is cool. But msgpack is also pretty nice.

Example: you work on the mobile team, the backend team is large and focuses on serving the web app, they send huge JSON payloads that the mobile app only partially need, and asking the backend team to now also serve msgpack is out of the question as things together with the backend and web teams were proven to be a PITA. In this scenario, writing a new, or bundling someone else's json library can significantly improv…

That’s an issue with your teams rather than the message format.

Even sending data that the mobile app doesn’t need raises flags.

Re: On-demand JSON: A better way to parse documents?

#43

Post by lpapez »

serial_dev wrote: post #29
eternityforest wrote: post #19

Why not just use msgpack? The advantage of JSON is that support is already built in to everything and you don't have to think about it. If you start having to actually make an effort to fuss with it, then why not consider other formats? This does have nice backwards compatibility with existing JSON stuff though, and sticking to standards is cool. But msgpack is also pretty nice.

Example: you work on the mobile team, the backend team is large and focuses on serving the web app, they send huge JSON payloads that the mobile app only partially need, and asking the backend team to now also serve msgpack is out of the question as things together with the backend and web teams were proven to be a PITA. In this scenario, writing a new, or bundling someone else's json library can significantly improv…

Sounds like a dysfunctional organization to be honest. Why couldn't you agree on the contents of the mobile API? If the backend team is slow, why can't the mobile team implement a proxy towards the backend to serve only the data they need?

Re: On-demand JSON: A better way to parse documents?

#44

Post by basil-rash »

zerocrates wrote: post #11
basil-rash wrote: post #9

> The JSON syntax is nearly a strict subset of the popular programming language JavaScript. What JSON isn’t valid JS?

The one thing I've seen mentioned before is the use of "__proto__" as a object property key. Though it's valid syntax in both JSON and JS like any other string key, it somewhat uniquely does something different if interpreted as JS (setting the created object's prototype) than it does if interpreted as JSON.

That's fair, though somewhat benign barring a prototype pollution vulnerability. The object still behaves the same as it would had you JSON.parse'd the same string (Object.getPrototypeOf aside).

Re: On-demand JSON: A better way to parse documents?

#45

Post by serial_dev »

lpapez wrote: post #43
serial_dev wrote: post #29

Earlier quoted context omitted.

Example: you work on the mobile team, the backend team is large and focuses on serving the web app, they send huge JSON payloads that the mobile app only partially need, and asking the backend team to now also serve msgpack is out of the question as things together with the backend and web teams were proven to be a PITA. In this scenario, writing a new, or bundling someone else's json library can significantly improv…

Sounds like a dysfunctional organization to be honest. Why couldn't you agree on the contents of the mobile API? If the backend team is slow, why can't the mobile team implement a proxy towards the backend to serve only the data they need?

I totally agree, it was a dysfunctional org, but sometimes fixing the org is extremely hard and you want to make progress today focusing on what you can do to improve the user's experience, instead of focusing on what others should change and do.

Re: On-demand JSON: A better way to parse documents?

#46

Post by klabb3 »

benatkin wrote: post #26
eternityforest wrote: post #19

Why not just use msgpack? The advantage of JSON is that support is already built in to everything and you don't have to think about it. If you start having to actually make an effort to fuss with it, then why not consider other formats? This does have nice backwards compatibility with existing JSON stuff though, and sticking to standards is cool. But msgpack is also pretty nice.

This seems to be geared towards using a heavily adopted format. Some would want to move to binary, but it's hard to find an ideal universal binary format. msgpack doesn't support a binary chunk bigger than 4gb, which is unfortunate. Also the JavaScript library doesn't handle Map vs plain object. In JSON you could have a 10GB Base64 blob, such as a video, in a string, no problem (from the format side, with a library Y…

> msgpack doesn't support a binary chunk bigger than 4gb, which is unfortunate.

I don’t care what behemoths people store in the formats they use but at the point you exceed “message size” the universality of any format is given up on. (Unless your format is designed to act as a database, like say a SQLite file.)

> In JSON you could have a 10GB Base64 blob, such as a video, in a string, no problem

Almost every stdlib json parser would choke on that, for good reason. Once you start adding partiality to a format, you get into tradeoffs with no obvious answers. Streaming giant arrays of objects? Scanning for keys in a map? How to validate duplicate keys without reading the whole file? Heck, just validating generally is now a deferred operation. Point is, it opens up a can of worms, where people argue endlessly about which use-cases are important, and meanwhile interop goes down the drain.

By all means, the stateful streaming / scanning space is both interesting and underserved. God knows we need something. Go build one, perhaps json can be used internally even. But cramming it all inside of json (or any message format) and expecting others to play along is a recipe for (another) fragmentation shitshow, imo.

Re: On-demand JSON: A better way to parse documents?

#47

Post by da_chicken »

bemusedthrow75 wrote: post #38
da_chicken wrote: post #33

Earlier quoted context omitted.

JSON has a lot more optimization that XML never got. Which I think says more about general interest in XML more than anything. Even today my experience is that XML processing varies wildly from "perfectly reasonable" to "maybe I can just do this with regex instead" even with widely used parsers. Also XML has a number of features to care about like attributes as well as elements, and also potentially about schema. It'…

> That is, if ` ` is always considered malformed, then why isn't the syntax simply ` `? XML isn't just a structured data format where close tags always run up against each other and whitespace is insignificant. It's also a descriptive document format which is often hand-authored. I think the argument is that the close tags being named makes those documents easier for a human author to understand. It certainly is my e…

No, I think that's true only in theory. It's true only in the hypothetical. It's only true when you're literally authoring the markup languages and no tools exists yet.

In the vast majority of cases, XML, like YAML or JSON, is machine written and machine parsed. Further, there's an almost unlimited number of tools available for manipulation. That's why nobody makes markup languages like SGML anymore unless they have to.

Heck, there's LaTeX, a document markup language which the HN users themselves seem to insist is incredibly easy to write in a text editor by hand, and that doesn't have verbose closing tags. Nevermind programming languages, etc.

No, SGML and it's descendants are weird in their insistence that structures must be as verbose as possible.

Re: On-demand JSON: A better way to parse documents?

#48

Post by anonyme-honteux »

bemusedthrow75 wrote: post #38
da_chicken wrote: post #33

Earlier quoted context omitted.

JSON has a lot more optimization that XML never got. Which I think says more about general interest in XML more than anything. Even today my experience is that XML processing varies wildly from "perfectly reasonable" to "maybe I can just do this with regex instead" even with widely used parsers. Also XML has a number of features to care about like attributes as well as elements, and also potentially about schema. It'…

> That is, if ` ` is always considered malformed, then why isn't the syntax simply ` `? XML isn't just a structured data format where close tags always run up against each other and whitespace is insignificant. It's also a descriptive document format which is often hand-authored. I think the argument is that the close tags being named makes those documents easier for a human author to understand. It certainly is my e…

Nobody has troubles reading a declarative DSL like

Tag1 { Tag2 { } }

Re: On-demand JSON: A better way to parse documents?

#49

Post by shadowbanned4 »

benatkin wrote: post #26
eternityforest wrote: post #19

Why not just use msgpack? The advantage of JSON is that support is already built in to everything and you don't have to think about it. If you start having to actually make an effort to fuss with it, then why not consider other formats? This does have nice backwards compatibility with existing JSON stuff though, and sticking to standards is cool. But msgpack is also pretty nice.

This seems to be geared towards using a heavily adopted format. Some would want to move to binary, but it's hard to find an ideal universal binary format. msgpack doesn't support a binary chunk bigger than 4gb, which is unfortunate. Also the JavaScript library doesn't handle Map vs plain object. In JSON you could have a 10GB Base64 blob, such as a video, in a string, no problem (from the format side, with a library Y…

[dead]

Re: On-demand JSON: A better way to parse documents?

#50

Post by creatonez »

fanseepawnts wrote: post #12

Sorry, I would never use this. Before I consume any json from any source or for any purpose I validate it. Lazy loading serves no purpose if you need validation. Hint: you need validation.

The purpose seems to be for when parsing a JSON document where large parts of it are irrelevant to your use case. It seems cursed, but it seems that using this method you can leave lee-way for unparsed parts and still validate what you are actually using, if the parser is robust enough.
Post reply on HN