Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parsing JSON is a Minefield

seriot.ch

Post reply on HN
Print view View original

81–90 of 257 posts

Re: Parsing JSON is a Minefield

#81

Post by tetha »

eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

Correct XML parsing carries at least 1 DoS attack, 1 serverside reflection attack, and those are just the two obvious ones. Hence, any secure XML endpoint must not be conform. That's a pretty nasty situation.

And I'm still traumatized from a university project in which we tried to compile XSD into serializer/deserializer pairs in C and java. The compiler structure was easy, code generation was easy, end2end tests cross-language with on-the-fly compilation was a little tricky because we had to hook up gcc / javac in a junit runner. But XSD simple types are hell, and XSD complex types are worse.

Re: Parsing JSON is a Minefield

#82

Post by taeric »

shagie wrote: post #64
taeric wrote: post #34

Earlier quoted context omitted.

Do you have a survey or other citation for it being a bad idea? I get that it enables bad behavior, per see. However, the idea of rejecting a customer/client because they did not form their request perfectly seems rather anti customer. Ideally, you'd both accept and correct. But that is the idea, just reworded.

The Harmful Consequences of Postel's Maxim - https://tools.ietf.org/html/draft-thomson-postel-was-wrong-0... (HN from 2015 https://news.ycombinator.com/item?id=9824638 ) Wrestling with Postel’s Law https://techblog.workiva.com/tech-blog/wrestling-postel’s-la...

The discussion was fun. And seems evenly split, at a quick reading.

More, I think it split on how you read it. If you view it as an absolute maxim to excuse poor implementations, it is panned. If you view it as a good faith behavior not to choke on the first mistake, you probably like it.

This is akin to grammar police. In life encounters, there is no real place for grammar policing. However, you should try to be grammatically correct.

Re: Parsing JSON is a Minefield

#83

Post by tenken »

inglor wrote: post #18

> For instance, RFC 8259 mentions that a design goal of JSON was to be "a subset of JavaScript", but it's actually not. Actually, it's really close https://github.com/tc39/proposal-json-superset This is a stage 3 proposal likely to make it to the next version of the spec. At which point JSON would truly be a subset of JavaScript.

Better late than never ...

Re: Parsing JSON is a Minefield

#84

Post by taeric »

Boulth wrote: post #77
taeric wrote: post #34

Earlier quoted context omitted.

Do you have a survey or other citation for it being a bad idea? I get that it enables bad behavior, per see. However, the idea of rejecting a customer/client because they did not form their request perfectly seems rather anti customer. Ideally, you'd both accept and correct. But that is the idea, just reworded.

Consumer is interested in fulfilling their need so they will fix their request so that it gets processed.

Or they will pick a service that works with them. This is literally how Google won most of their market share. Sure, there used to be a bit of syntax on the search, but Google always had a single input field and did not barf requests back to users because they put a field in the wrong input.

Re: Parsing JSON is a Minefield

#85

Post by digi_owl »

bhldr wrote: post #20
0xcde4c3db wrote: post #15

Earlier quoted context omitted.

> "be liberal in what you accept, and be conservative in what you send" This is commonly known as Postel's Law, and comes from one of the TCP RFCs [1]. [1] https://en.wikipedia.org/wiki/Robustness_principle

This is also widely considered a bad idea now. Making liberal consumers allows for sloppy producers. Over time this requires new consumers to conform to these sloppy producers to maintain compatibility. Just look at the clusterfuck that HTML5 has become. You need to have extremely deep pockets to enter that market.

The HTML5 clusterfuck comes from having the biggest players being allowed to adjust the goal as they see fit, when they see fit (aka "living document").

Re: Parsing JSON is a Minefield

#86

Post by tannhaeuser »

teddyh wrote: post #48
eponeponepon wrote: post #41

Earlier quoted context omitted.

Just for the record - XML and HTML are both subsets of SGML, somewhat overlapping, but by no means coterminous with each other (at least until HTML 5 - I'm honestly not sure what it's relationship to SGML is). And, speaking from experience, the XML nay-sayers should largely be glad if they never had to deal with SGML :)

HTML pretended to be a subset of SGML, but never really was, and the illusion quickly dispersed as time went on, since HTML was strictly pragmatic and ran in resource-constrained environments (the desktop), while SGML was academic, largely theoretical, and ran on servers, analyzing text. XML, on the other hand, was more of a back-formation – a generalization of HTML; it was not, as I understand it, directly related t…

> XML [...] was not [...] directly related to SGML in any way

That's incorrect. XML is by definition a proper subset of WebSGML, the SGML revision specified in ISO 8879:1986 Annex K. These two specifications were published around the same time and authored by the same people.

In a nutshell, XML added DTD-less SGML (eg. such that every document can be parsed without markup declarations, unlike eg. HTML which has `img` and other empty elements the parser needs to know about) and XML-style empty elements. The features removed from SGML to become XML were tag inference/omission (as used in HTML), short references (for things such as Wiki syntax, CSV, and even JSON parsing), uses of marked sections other than `CDATA`, more complex use cases for notations, and link process declarations ("stylesheets") plus a couple others.

Re: Parsing JSON is a Minefield

#87

Post by Someone »

guntars wrote: post #57
Someone wrote: post #17

Earlier quoted context omitted.

If your API takes json input, some of those issues are potential security or DoS issues. For example, if you validate your json in your web front-end (EDIT: I used the wrong term. What I meant here is the server-side process that’s in front of your database) and then pass the string received to your json-aware database, you’re likely using two json implementations that may have different ideas about what constitutes…

I sure hope you don’t just put random user provided blobs in your database, even if they’re validated. Also, how do you validate without parsing? If it’s parsed, might as well serialize again when saving to the DB.

”If it’s parsed, might as well serialize again when saving to the DB”

You didn’t grow up in the 1980’s, I guess :-)

Why spend cycles serializing again if you already have that string?

Re: Parsing JSON is a Minefield

#88

Post by koolba »

3pt14159 wrote: post #19
eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

Disagree. I can always make my JSON act like XML if I want to. When I'm following something like JSON API v1.1 I get a lot of the advantages that I'd get from XML with 99% less bloat. You want types? Go for it! There are even official typed JSON options out there. The security / parsing issues with XML alone are enough for me to rule it out. How many critical security issues are the result of libxml? Nokogiri / libxm…

> There are even official typed JSON options out there.

What are the "official" ones?

Everything I've seen involves validation and explicit formatting for a couple specific types (ex: ISO-8601 dates) but it requires the target to specify what it expects.

There's no way to tell staring at a JSON string if "2018-04-22" is meant to be a date rather than a text string.

Post reply on HN