Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parsing JSON is a Minefield

seriot.ch

Post reply on HN
Print view View original

121–130 of 257 posts

Re: Parsing JSON is a Minefield

#121

Post by pjscott »

mixedCase wrote: post #8
eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

If I needed such strictness in parsing I'd fall back to s-expressions, not something that requires a parser like this: $ ls -lah /usr/lib/libxml2.so.2.9.8 -rwxr-xr-x 1 root root 1,4M mar 27 17:46 /usr/lib/libxml2.so.2.9.8

It's worth pointing out that libxml2 also contains an HTML parser, implementations of XPath, XPointer, various other half-forgotten things beginning with the letter X, a Relax-NG implementation, and clients for both HTTP and FTP. The actual XML parser doesn't need any of that, and almost certainly takes up a lot less than 1.4 MB.

Re: Parsing JSON is a Minefield

#122

Post by eropple »

drawkbox wrote: post #107
eponeponepon wrote: post #4

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

Vogons destroyed XML and they would love to destroy JSON. Back away from the JSON vogons, go make another 'simple' format that you put in all your edge cases for complexity. Just try to make a format more simple than JSON, it is based off the basic object, list and basic types string, number, date, bool etc. Where data doesn't fit in those you make it fit or move to another format like YAML, BSON, XML, binary standar…

If other people suggesting that, hey, maybe we should actually be able to express a number correctly makes you splutter about "vogons" or whatever, perhaps it is not they who should take a step back. (For this isn't just "massive" numbers, but anything that isn't a float--themselves ranking just after `null` as the worst disaster in current use in general-purpose programming.)

Telling people to "just" take actions that decrease the reliability and the rigor of their data because of...vogons?...is one of those weird middlebrow things that HN tends to try to steer clear of, last I checked.

(edit: To be clear, I get the reference, I think it's a silly one both for the childish regard the poster to whom I am replying has for other people and textually because it doesn't even hang.)

Re: Parsing JSON is a Minefield

#123

Post by ris »

orf wrote: post #101
ris wrote: post #78

Earlier quoted context omitted.

> the whitespace issues of YAML It's not just whitespace issues yaml has - try storing ISO two letter country codes as values and as soon as you get to norway, you've got a boolean. There are many things which are deceptively "simple" where that actually means "I haven't thought about this very much".

Could you not just use 'no'?

Absolutely, but only once you've tracked down the unexpected bug.

Re: Parsing JSON is a Minefield

#124

Post by gnud »

cptn_brittish wrote: post #120
icebraining wrote: post #100

Earlier quoted context omitted.

I'm not saying it can't be mapped; I'm saying it loses semantics in the translation. For example, how do you represent a boolean in a s-exp, such as that anyone with "the s-exp spec" can unambiguously know that's a boolean?

You use p-lists instead so that {"foo":"bar"} becomes ( :foo "bar")

Won't you inherit all the encoding trouble this article goes on about? As well as the problems with integer precision? And get a bunch of new ones with special characters in keys...

I'm sure you _could_ specify a nice sexpr format. I'm not sure the specification would be simple though. And just saying "use sexprs" leaves you with all the problems you have when you say "use json".

Re: Parsing JSON is a Minefield

#125

Post by still_grokking »

taeric wrote: post #82
shagie wrote: post #64

Earlier quoted context omitted.

The Harmful Consequences of Postel's Maxim - https://tools.ietf.org/html/draft-thomson-postel-was-wrong-0... (HN from 2015 https://news.ycombinator.com/item?id=9824638 ) Wrestling with Postel’s Law https://techblog.workiva.com/tech-blog/wrestling-postel’s-la...

The discussion was fun. And seems evenly split, at a quick reading. More, I think it split on how you read it. If you view it as an absolute maxim to excuse poor implementations, it is panned. If you view it as a good faith behavior not to choke on the first mistake, you probably like it. This is akin to grammar police. In life encounters, there is no real place for grammar policing. However, you should try to be gra…

> This is akin to grammar police. In life encounters, there is no real place for grammar policing. However, you should try to be grammatically correct.

That's because most humans have feelings. But most machines don't. So that's not comparable.

Re: Parsing JSON is a Minefield

#126

Post by iainmerrick »

pjscott wrote: post #117
izacus wrote: post #79

Earlier quoted context omitted.

How do Protocol Buffers (which I see used quite alot in similar environments as JSON) compare? Anyone has experience in the format?

I wrote a protobuf decoder once and found it to be remarkably pleasant. Getting the decoder working only took a few hours. The format was obviously designed to be straightforward -- no escaping, no backtracking, no ambiguity. I believe the grammar is LL(0), which is a nice touch. And because it's not meant to be human-readable, there's no incentive for people to make their parsers deviate from the strict grammar; e.g…

There is no such quirk. Trailing commas are simply not standard JSON (sadly).

Even if they were, it would still be easy to parse!

Re: Parsing JSON is a Minefield

#127

Post by freshhawk »

It would be great if programmers learned from markdown and json.

Here is the lesson:

1. We need something simpler, so I will make a simple solution to this problem 2. Simple should also mean no strict spec, support for versioning or any of those engineer things. All that engineer shit is boring and I can tell myself this laziness is "staying simple" 3. OH SHIT, I was totally right about #1 so this got popular and having been designed as just a toy is causing a lot of problems for a massive number of people ... now my incompetence regarding #2 is on display and there is nothing I can do about it

I'm not saying "Thou shalt always add bureaucracy to your toy projects", but look at what happened and think about how Gruber and Crawford will be remembered, partly if not mostly, for being "the asshole who screwed up X". If you go the other way programmers will think "damn I hate these RFCs, these suits are messing up the beautiful vision of Saint [your name here]".

Re: Parsing JSON is a Minefield

#128

Post by mchanson »

freshhawk wrote: post #127

It would be great if programmers learned from markdown and json. Here is the lesson: 1. We need something simpler, so I will make a simple solution to this problem 2. Simple should also mean no strict spec, support for versioning or any of those engineer things. All that engineer shit is boring and I can tell myself this laziness is "staying simple" 3. OH SHIT, I was totally right about #1 so this got popular and hav…

Oh yes RFC process always keeps things from having compatibility issues.

I definitely never saw any issues with all those XML based standards like SOAP or XSLT.

Re: Parsing JSON is a Minefield

#129

Post by pjscott »

iainmerrick wrote: post #126
pjscott wrote: post #117

Earlier quoted context omitted.

I wrote a protobuf decoder once and found it to be remarkably pleasant. Getting the decoder working only took a few hours. The format was obviously designed to be straightforward -- no escaping, no backtracking, no ambiguity. I believe the grammar is LL(0), which is a nice touch. And because it's not meant to be human-readable, there's no incentive for people to make their parsers deviate from the strict grammar; e.g…

There is no such quirk. Trailing commas are simply not standard JSON (sadly). Even if they were, it would still be easy to parse!

You're right in the same technically-correct sense that parsing HTML 4 is easy: it's just SGML! And as long as you don't have to support any of the crazy deviations from the spec that some people have come to rely on, that's fine.

There's a whole spectrum of unofficial parser "helpfulness" here, with HTML 4 being an extreme case of parsers filled with hacks to deal with existing broken data, protobufs being an extreme case of parsers doing the One and Only True Thing, and JSON mostly toward the same end of the spectrum as protobufs, but a bit less so.

Re: Parsing JSON is a Minefield

#130

Post by k__ »

Reminds me of some JSON I got from an API.

It was always malformed and I always wrote the dev that he should fix it.

He always did, but every new endpoint was malformed again.

One day I looked at the code and it was full of string concatinations of DB results...

Post reply on HN