Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parsing JSON is a Minefield

seriot.ch

Post reply on HN
Print view View original

21–30 of 257 posts

Re: Parsing JSON is a Minefield

#21

Post by borplk »

I think the idea of humans sharing a language with computers is problematic at a fundamental level.

(the whole $dataformat "easy to read for humans")

It becomes a source of never ending lose-lose compromises where the more points you give to human convenience the more points you take away from machine convenience and vice versa.

Then you end up having to "settle" for something in between that is just ambiguous and problematic for machines to be able to deal with it and just noisy enough for humans to be able to cope looking at it and edit it. That's basically what JSON is.

If we accept to use a transformation step and better tooling we can free the representation from this tension of "friendly for computers vs friendly for humans".

It's also a bit odd that we apply this readability obsession only to these data formats.

I don't hear people wanting a human readable text representation of their audio, video or images.

Re: Parsing JSON is a Minefield

#22

Post by bhldr »

mixedCase wrote: post #8
eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

If I needed such strictness in parsing I'd fall back to s-expressions, not something that requires a parser like this: $ ls -lah /usr/lib/libxml2.so.2.9.8 -rwxr-xr-x 1 root root 1,4M mar 27 17:46 /usr/lib/libxml2.so.2.9.8

It's easy to be strict if you are both a producer and a consumer.

Re: Parsing JSON is a Minefield

#23

Post by looperhacks »

3pt14159 wrote: post #19
eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

Disagree. I can always make my JSON act like XML if I want to. When I'm following something like JSON API v1.1 I get a lot of the advantages that I'd get from XML with 99% less bloat. You want types? Go for it! There are even official typed JSON options out there. The security / parsing issues with XML alone are enough for me to rule it out. How many critical security issues are the result of libxml? Nokogiri / libxm…

HTML isn't XML. It's close, but it isn't. There's XHTML for that.

Re: Parsing JSON is a Minefield

#24

Post by ajross »

borplk wrote: post #21

I think the idea of humans sharing a language with computers is problematic at a fundamental level. (the whole $dataformat "easy to read for humans") It becomes a source of never ending lose-lose compromises where the more points you give to human convenience the more points you take away from machine convenience and vice versa. Then you end up having to "settle" for something in between that is just ambiguous and pr…

Nothing in your argument seems relevant to whether your mythical format is "readable" or not. The industry has had extensive experience dealing with interchange of commonly-used binary formats (.xls comes to mind as one that still seems some use) and if anything that experience was even worse.

You're just saying that if we stopped trying to adhere to "our" aesthetic and replaced it with "your" aesthetic when we created an all-new format, that it would be better. I think there's an XKCD to cite about that...

Re: Parsing JSON is a Minefield

#25

Post by icebraining »

mixedCase wrote: post #8
eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

If I needed such strictness in parsing I'd fall back to s-expressions, not something that requires a parser like this: $ ls -lah /usr/lib/libxml2.so.2.9.8 -rwxr-xr-x 1 root root 1,4M mar 27 17:46 /usr/lib/libxml2.so.2.9.8

But that's the point, isn't it? S-expressions are light because they define very little, it's only a tree of undefined blobs of data (atoms). It's even more limited than JSON.

Re: Parsing JSON is a Minefield

#26

Post by borplk »

ajross wrote: post #24
borplk wrote: post #21

I think the idea of humans sharing a language with computers is problematic at a fundamental level. (the whole $dataformat "easy to read for humans") It becomes a source of never ending lose-lose compromises where the more points you give to human convenience the more points you take away from machine convenience and vice versa. Then you end up having to "settle" for something in between that is just ambiguous and pr…

Nothing in your argument seems relevant to whether your mythical format is "readable" or not. The industry has had extensive experience dealing with interchange of commonly-used binary formats (.xls comes to mind as one that still seems some use) and if anything that experience was even worse. You're just saying that if we stopped trying to adhere to "our" aesthetic and replaced it with "your" aesthetic when we creat…

You have misunderstood the point I was trying to make.

I'm saying the idea that we as humans should view/consume the exact same thing raw and byte-by-byte without any transformations in between as computers do creates a problematic trade-off because that representation now has two very different masters, humans and machines.

If you want to make that representation pretty and friendly for humans it becomes hard to parse and consume for machines.

If you try to make it friendly for machines it becomes inaccessible, noisy and annoying for humans.

So you have to settle somewhere in between that is tolerable for both masters but not perfect for either one.

With some exaggeration, it's as if we required the binary representation of an image to be "raw viewable/consumable" by humans in a text editor.

Re: Parsing JSON is a Minefield

#28

Post by ProblemFactory »

jstimpfle wrote: post #7
eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

Xml sort-of died (in many domains) because of its insane complexity and its redundant ways of specifying relations (child relationships vs explicit relationships, tag name data, attributes data, body data), lack of legibility, parser performance (probably an inherent problem due to hierarchical representation?) and other issues like even meaning of whitespace. >50% of the JSON or Xml I've seen would actually be much…

XML parsers are necessarily over-complicated for structured data, because it is a text markup language, not a nested data structure language.

123 Hello World Road, 12345, CA

is perfectly sensible XML. The address is not a tree structure or a key-value dictionary - it is free text with optional markup for some words.

You can use XML to represent nested data structures with lists and dictionaries, but the parsers and their public APIs must still handle the freeform text case as well.

Re: Parsing JSON is a Minefield

#29

Post by rhapsodic »

bhldr wrote: post #20
0xcde4c3db wrote: post #15

Earlier quoted context omitted.

> "be liberal in what you accept, and be conservative in what you send" This is commonly known as Postel's Law, and comes from one of the TCP RFCs [1]. [1] https://en.wikipedia.org/wiki/Robustness_principle

This is also widely considered a bad idea now. Making liberal consumers allows for sloppy producers. Over time this requires new consumers to conform to these sloppy producers to maintain compatibility. Just look at the clusterfuck that HTML5 has become. You need to have extremely deep pockets to enter that market.

> Just look at the clusterfuck that HTML5 has become. You need to have extremely deep pockets to enter that market.

What do you mean by "enter that market"?

Re: Parsing JSON is a Minefield

#30

Post by hsivonen »

This document is missing the XMLHttpRequest/Fetch JSON profile. ECMA operates on a sequence of Unicode code points (really code points, not scalar values!). WHATWG defines how you go from bytes over HTTP to something you can pass to the ECMA-specified JSON parser.
Post reply on HN