Live data from Hacker News

Parsing JSON is a Minefield

seriot.ch

241–250 of 257 posts

Re: Parsing JSON is a Minefield

#241

Earlier quoted context omitted.

HTML isn't XML. It's close, but it isn't. There's XHTML for that.

Just for the record - XML and HTML are both subsets of SGML, somewhat overlapping, but by no means coterminous with each other (at least until HTML 5 - I'm honestly not sure what it's relationship to SGML is). And, speaking from experience, the XML nay-sayers should largely be glad if they never had to deal with SGML :)

http://sgmljs.net/docs/html5.html

HTML5 is almost a subset of SGML, barring some ambiguities in itz table spec, HTML comments in script tags and the spellcheck and contenteditable attributes.

Re: Parsing JSON is a Minefield

#242

Earlier quoted context omitted.

From someone who has written TeX macro’s before: you probably mistake the ‘clean’ environment of LaTeX with the core TeX language. The former is reasonable, if very limited, the latter is die-hard “you thought you knew how to program, but this proves you wrong”-material. XML over TeX any time and LISP-like over XML (with structural macros)

I would make a subtle adjustment there: "you thought Knuth knew how to design, this proves you wrong". \makeatspecial here we go\oh\boy ...

Ah, yes, agreed! That was my first thought as well when I went into the deep flaming pit of TeX macros. Nevertheless, I guess he didn't have much reference material back then or did he?

Re: Parsing JSON is a Minefield

#243

Earlier quoted context omitted.

I would make a subtle adjustment there: "you thought Knuth knew how to design, this proves you wrong". \makeatspecial here we go\oh\boy ...

Ah, yes, agreed! That was my first thought as well when I went into the deep flaming pit of TeX macros. Nevertheless, I guess he didn't have much reference material back then or did he?

FWIW, I remember a quote about Knuth and TeX, "He tried really not to make it a programming language, but ultimately he failed". I don't remember who said that.

Re: Parsing JSON is a Minefield

#244

Earlier quoted context omitted.

Why in the world would I write a parser in bash or awk, regardless of the format? I certainly have better things to do. It doesn't matter how sensible a format is, those tools are simply not appropriate to write a parser in. I have, however, written a JSON parser before at a previous company in C++ (they didn't want to pull in a library). It wasn't particularly hard. And yes, like any other parser, it accepted additi…

Why in the world would I have to use JavaScript or any object oriented programming language just because the application is poorly thought out and emits JSON? I certainly have better things to do. It doesn't matter how sensible a format is, those tools are simply not appropriate to write a parser in. AWK is a language designed for data parsing and processing. That is what it is designed to do. How did you solve the p…

Hope one of these helps with your problem:

https://github.com/step-/JSON.awk

https://github.com/dominictarr/JSON.sh

Re: Parsing JSON is a Minefield

#245
post #36

This is the one thing that the JSON-against-XML holy warriors need to understand properly. Yes, JSON's less verbose; yes, it's just "plain text" (in as much as there is such a thing); yes, XML makes you put closing tags in - but if you need reliable parsing and rock-solid specifications (and it's reasonably likely that you do, even if you think you don't...), then XML, for all its faults, is very likely the better wa…

that point is moot with xml, since xml (and don't even get me started on xslt) is as much of a mess as json. Go try to nest a few XML documents and then came back saying if you still love it, or even if you still consider it a well-define standard.

I’m very confused. What’s hard about nesting XML documents? That’s one of XML’s strengths, and JSON’s weaknesses, because JSON doesn’t have namespaces.

Re: Parsing JSON is a Minefield

#246

Earlier quoted context omitted.

Ah, yes, agreed! That was my first thought as well when I went into the deep flaming pit of TeX macros. Nevertheless, I guess he didn't have much reference material back then or did he?

FWIW, I remember a quote about Knuth and TeX, "He tried really not to make it a programming language, but ultimately he failed". I don't remember who said that.

You're probably thinking of Knuth himself. He's mentioned several times how he never intended to make a programming language, and how puzzled he is that people write programs in TeX macros.

E.g.:

> In some sense I put in many of the programming features kicking and screaming [...] Every system I looked at had its own universal Turing machine built into it somehow, and everybody’s was a little different from everybody else’s. So I thought, “Well, I’m not going to design a programming language; I wanted to have just a typesetting language.” Little by little, I needed more features and so the programming constructs grew.[...] as a programmer, I was tired of having to learn ten different almost-the-same programming languages for every system I looked at; I was going to try to avoid that.

etc. (https://www.ntg.nl/maps/16/15.pdf)

and:

> I was really thinking of TeX as something that the more programming it had in it, the less it was doing its real mission of typesetting. When I put in the calculation of prime numbers into the TeX manual I was not thinking of this as the way to use TeX. I was thinking, "Oh, by the way, look at this: dogs can stand on their hind legs and TeX can calculate prime numbers."

(Coders at Work interview)

In fact, if you use TeX the way Knuth intended and uses it, then the use of macros or programming is really quite minimal. It's only LaTeX that to pursue a better document interface for the user, ends up writing horrifically complex macros -- Mittelbach mentions that nine out of ten "dirty tricks" mentioned by Knuth in the TeXbook are actually used in the source code of LaTeX!

Re: Parsing JSON is a Minefield

#247

Earlier quoted context omitted.

Why in the world would I write a parser in bash or awk, regardless of the format? I certainly have better things to do. It doesn't matter how sensible a format is, those tools are simply not appropriate to write a parser in. I have, however, written a JSON parser before at a previous company in C++ (they didn't want to pull in a library). It wasn't particularly hard. And yes, like any other parser, it accepted additi…

Why in the world would I have to use JavaScript or any object oriented programming language just because the application is poorly thought out and emits JSON? I certainly have better things to do. It doesn't matter how sensible a format is, those tools are simply not appropriate to write a parser in. AWK is a language designed for data parsing and processing. That is what it is designed to do. How did you solve the p…

Json is clearly data.

If awk cannot parse it, perhaps it’s a shortcoming of awk? =)

Re: Parsing JSON is a Minefield

#248

Earlier quoted context omitted.

Why in the world would I have to use JavaScript or any object oriented programming language just because the application is poorly thought out and emits JSON? I certainly have better things to do. It doesn't matter how sensible a format is, those tools are simply not appropriate to write a parser in. AWK is a language designed for data parsing and processing. That is what it is designed to do. How did you solve the p…

Json is clearly data. If awk cannot parse it, perhaps it’s a shortcoming of awk? =)

It’s not an AWK limitation; recursive descent parsers are non-trivial to implement and tricky to get right.

Re: Parsing JSON is a Minefield

#249

Earlier quoted context omitted.

I consider it more like behaviour outside of the spec is undefined, and some parsers have bugs.

A good spec should define all those corner cases. With Json, it's possible for two bug free libraries to take the same input and produce a different output.

Only if the input is invalid JSON.

Re: Parsing JSON is a Minefield

#250

Earlier quoted context omitted.

JSON parsers all agree what JSON actually is (except for the brown colored fields in the compat matrix, which are actual JSON compat bugs). JSON parsers may also permit various additional variations which are very explicitly not JSON. This means that they may accept a handcrafted, technically invalid JSON document. However, a JSON encoder may never generate a document containing such "extensions", as this would not b…

> As the robustness principle goes [...] "The Harmful Consequences of the Robustness Principle" https://tools.ietf.org/html/draft-thomson-postel-was-wrong-0...

The author advocates adding previously invalid extensions to the spec as they come up, as an alternate way to handle 'robustness'. How would this not rapidly lead to a bloated, buggy, unmaintainable spec?
Post reply on HN