Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parsing JSON is a Minefield

seriot.ch

Post reply on HN
Print view View original

31–40 of 257 posts

Re: Parsing JSON is a Minefield

#31

Post by hyperdimension »

rhapsodic wrote: post #29
bhldr wrote: post #20

Earlier quoted context omitted.

This is also widely considered a bad idea now. Making liberal consumers allows for sloppy producers. Over time this requires new consumers to conform to these sloppy producers to maintain compatibility. Just look at the clusterfuck that HTML5 has become. You need to have extremely deep pockets to enter that market.

> Just look at the clusterfuck that HTML5 has become. You need to have extremely deep pockets to enter that market. What do you mean by "enter that market"?

I think they mean needing deep pockets to write a new browser, with all the complexity that modern HTML+JS entails.

Re: Parsing JSON is a Minefield

#32

Post by jstimpfle »

ProblemFactory wrote: post #28
jstimpfle wrote: post #7

Earlier quoted context omitted.

Xml sort-of died (in many domains) because of its insane complexity and its redundant ways of specifying relations (child relationships vs explicit relationships, tag name data, attributes data, body data), lack of legibility, parser performance (probably an inherent problem due to hierarchical representation?) and other issues like even meaning of whitespace. >50% of the JSON or Xml I've seen would actually be much…

XML parsers are necessarily over-complicated for structured data, because it is a text markup language, not a nested data structure language. 123 Hello World Road, 12345 , CA is perfectly sensible XML. The address is not a tree structure or a key-value dictionary - it is free text with optional markup for some words. You can use XML to represent nested data structures with lists and dictionaries, but the parsers and…

Yep, the application to text documents is valid in my eyes, as well. Although there are lighter weight and/or more extensible approaches, like TeX. (update, clarificaton: I mean just the markup syntax, not the compuational model)

Re: Parsing JSON is a Minefield

#33

Post by rurban »

Parsing JSON is not a minefield. It is technically trivial and pretty secure. Compared to other specs it's not that bad, but of course there are still some security concerns, esp. in the last two JSON RFC updates, which made it worse and not better.

But most other commonly uses transport formats are much worse, and much harder to parse. Start reading at http://search.cpan.org/~rurban/Cpanel-JSON-XS-4.02/XS.pm#RFC...

Re: Parsing JSON is a Minefield

#34

Post by taeric »

bhldr wrote: post #20
0xcde4c3db wrote: post #15

Earlier quoted context omitted.

> "be liberal in what you accept, and be conservative in what you send" This is commonly known as Postel's Law, and comes from one of the TCP RFCs [1]. [1] https://en.wikipedia.org/wiki/Robustness_principle

This is also widely considered a bad idea now. Making liberal consumers allows for sloppy producers. Over time this requires new consumers to conform to these sloppy producers to maintain compatibility. Just look at the clusterfuck that HTML5 has become. You need to have extremely deep pockets to enter that market.

Do you have a survey or other citation for it being a bad idea? I get that it enables bad behavior, per see. However, the idea of rejecting a customer/client because they did not form their request perfectly seems rather anti customer.

Ideally, you'd both accept and correct. But that is the idea, just reworded.

Re: Parsing JSON is a Minefield

#35

Post by jstimpfle »

rmrfrmrf wrote: post #14
eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

If you think XML doesn't suffer from all the same issues, you haven't used it enough. I'd use protobuf for something that needs stict serialization and parsing.

I think protobuf is a binary format though?

Re: Parsing JSON is a Minefield

#36

Post by gcb0 »

eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

that point is moot with xml, since xml (and don't even get me started on xslt) is as much of a mess as json.

Go try to nest a few XML documents and then came back saying if you still love it, or even if you still consider it a well-define standard.

Re: Parsing JSON is a Minefield

#37

Post by kbouck »

0xcde4c3db wrote: post #15
kcolford wrote: post #13

This brings to mind the old internet motto (someone correct me on the actual source): "be liberal in what you accept, and be conservative in what you send". JSON is pretty clear on what certain things should mean, strings are Unicode plus escape sequences, objects map keys to values, arrays are ordered collections of values, the whole serialized payload should be Unicode, etc. Even those things can be relaxed further…

> "be liberal in what you accept, and be conservative in what you send" This is commonly known as Postel's Law, and comes from one of the TCP RFCs [1]. [1] https://en.wikipedia.org/wiki/Robustness_principle

Here are some thoughtful arguments against "Postel's Law":

https://tools.ietf.org/html/draft-thomson-postel-was-wrong-0...

Re: Parsing JSON is a Minefield

#38

Post by ajross »

borplk wrote: post #26
ajross wrote: post #24

Earlier quoted context omitted.

Nothing in your argument seems relevant to whether your mythical format is "readable" or not. The industry has had extensive experience dealing with interchange of commonly-used binary formats (.xls comes to mind as one that still seems some use) and if anything that experience was even worse. You're just saying that if we stopped trying to adhere to "our" aesthetic and replaced it with "your" aesthetic when we creat…

You have misunderstood the point I was trying to make. I'm saying the idea that we as humans should view/consume the exact same thing raw and byte-by-byte without any transformations in between as computers do creates a problematic trade-off because that representation now has two very different masters, humans and machines. If you want to make that representation pretty and friendly for humans it becomes hard to par…

No, I'm pretty sure I took that point. It's just that... that isn't the problem. All formats get messy for the same reason that all software designs get messy. It's just that messy software is amenable to replacement, while messy formats leave their garbage in public to scream about on HN.

And like I said, we've been where you want to be: .xls, in particular, is actually a very simple format at its core and easily inspectable with a hex dump or whatnot. Likewise Wordperfect's binary format back in the day was straightforward for humans to use, yet still binary. It didn't help. They still sucked.

To wit: you can't solve this problem your way. All you'll do is create another messy format, c.f. XKCD 927.

Re: Parsing JSON is a Minefield

#39

Post by keymone »

i wish there was a chance for EDN[1] to replace JSON. it's a shame the industry defaulted to a subset of javascript as a data notation format considering all it's shortcomings =/

yeah, i get it, "but it has native support in all browsers" is a valid argument, i just wish it wasn't.

[1] https://github.com/edn-format/edn

Re: Parsing JSON is a Minefield

#40

Post by helaan »

Someone wrote: post #17
JasonFruit wrote: post #2

This is interesting and important in one way: anything poorly specified will eventually cause a problem for someone, somewhere. That being said, my first response was to complete the title, ". . . yet it remains useful and nearly trouble-free in practice." There's a lot of, "You know what I mean!" in the JSON definition, but in most cases, we really do know what Crockford means.

If your API takes json input, some of those issues are potential security or DoS issues. For example, if you validate your json in your web front-end (EDIT: I used the wrong term. What I meant here is the server-side process that’s in front of your database) and then pass the string received to your json-aware database, you’re likely using two json implementations that may have different ideas about what constitutes…

Reminds me of last years CouchDB bug (CVE-2017-12635) which was caused by two JSON parsers disagreeing on duplicate keys: here it was possible to add a second key with user roles, allowing a user to give admin rights to itself. JSON parser issues are real.
Post reply on HN