Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parsing JSON is a Minefield

seriot.ch

Post reply on HN
Print view View original

141–150 of 257 posts

Re: Parsing JSON is a Minefield

#141

Post by kazinator »

edejong wrote: post #106
jstimpfle wrote: post #32

Earlier quoted context omitted.

Yep, the application to text documents is valid in my eyes, as well. Although there are lighter weight and/or more extensible approaches, like TeX. (update, clarificaton: I mean just the markup syntax, not the compuational model)

From someone who has written TeX macro’s before: you probably mistake the ‘clean’ environment of LaTeX with the core TeX language. The former is reasonable, if very limited, the latter is die-hard “you thought you knew how to program, but this proves you wrong”-material. XML over TeX any time and LISP-like over XML (with structural macros)

I would make a subtle adjustment there: "you thought Knuth knew how to design, this proves you wrong". \makeatspecial here we go\oh\boy ...

Re: Parsing JSON is a Minefield

#142

Post by saagarjha »

bastawhiz wrote: post #47
rainbowmverse wrote: post #43

Earlier quoted context omitted.

>> I don't hear people wanting a human readable text representation of their audio, video or images. This is, in fact, a huge concern for people who think about accessibility.

Indeed! Facebook spends a ridiculous amount of resources creating text summaries of what's in user-provided images. Check out the alt="" tags next time you're scrolling through your feed. Every single image has one.

Of course, their methodology for doing so is nonstandard, since there are obvious incentives for Facebook to use it and keep it to themselves. Unfortunately this keeps these transformations out of reach for most applications.

Re: Parsing JSON is a Minefield

#143

Post by mjevans »

helaan wrote: post #40
Someone wrote: post #17

Earlier quoted context omitted.

If your API takes json input, some of those issues are potential security or DoS issues. For example, if you validate your json in your web front-end (EDIT: I used the wrong term. What I meant here is the server-side process that’s in front of your database) and then pass the string received to your json-aware database, you’re likely using two json implementations that may have different ideas about what constitutes…

Reminds me of last years CouchDB bug (CVE-2017-12635) which was caused by two JSON parsers disagreeing on duplicate keys: here it was possible to add a second key with user roles, allowing a user to give admin rights to itself. JSON parser issues are real.

Normalize before approval and add filters that only allow in /expressly approved/ items from insecure environments.

Re: Parsing JSON is a Minefield

#144

Post by mjevans »

Someone wrote: post #118
toast0 wrote: post #99

Earlier quoted context omitted.

Because experience has shown us that today's parsers don't detect tomorrow's 0-day parsing bugs; but serializing a clean version of what was parsed is more likely to be safe (see lots of jpeg, mpeg, etc exploits)

More likely, yes, but it need not help you here. Let’s say Chuck sends {“command”:”feed”, “command”:”kill”} Alice uses json parser #1. It keeps both “command” entries. Alice next checks the “command” value against a whitelist. Her json library reads the first value, returning the benign “feed”. Alice next serializes the parsed structure and sends it to Bob. The serializer she uses returns the exact string Eve sent. B…

Input validation, security, and message construction are all hard topics; particularly when combined.

Re: Parsing JSON is a Minefield

#145

Post by 3pt14159 »

looperhacks wrote: post #23
3pt14159 wrote: post #19

Earlier quoted context omitted.

Disagree. I can always make my JSON act like XML if I want to. When I'm following something like JSON API v1.1 I get a lot of the advantages that I'd get from XML with 99% less bloat. You want types? Go for it! There are even official typed JSON options out there. The security / parsing issues with XML alone are enough for me to rule it out. How many critical security issues are the result of libxml? Nokogiri / libxm…

HTML isn't XML. It's close, but it isn't. There's XHTML for that.

I’m aware that HTML 5 wasn’t an XML but I thought XHTML was an XML and browsers still have to support it because not everyone is on HTML 5.

Either way, my original draft included language around the distinction, but I felt I’d already written too much so I cut it.

Re: Parsing JSON is a Minefield

#146

Post by ospider »

skywhopper wrote: post #52

While this is true of JSON, it's also true of any other non-trivial serialization and/or encoding format. The main lessons to learn here are that: 1) implementation matters 2) "simple" specs never really are It's definitely important to have documents like this one that explore the edge cases and the differences between implementations, but you can replace "JSON" in the introductory paragraph with any other serializa…

Paring "csv" and "obj" format is also difficult. They both have "simple" specs, but neither of them has a standard spec.

Re: Parsing JSON is a Minefield

#147

Post by agrafix »

ori_b wrote: post #131
izacus wrote: post #79

Earlier quoted context omitted.

How do Protocol Buffers (which I see used quite alot in similar environments as JSON) compare? Anyone has experience in the format?

Protocol buffers sidestep the issue of independent implementations behaving differently by simply not having widely used independent implementations. That said, I've still been bitten by the Python implementation on the Mac acting differently from the C++ implementation on Linux, although I can't remember exactly what the issue was right now.

One of the things that has bitten me before was the maximum allowed nesting depth - this was different in two implementations, so one rejected the payload while the other one parsed it fine :(

Re: Parsing JSON is a Minefield

#148

Post by unmole »

mchanson wrote: post #128
freshhawk wrote: post #127

It would be great if programmers learned from markdown and json. Here is the lesson: 1. We need something simpler, so I will make a simple solution to this problem 2. Simple should also mean no strict spec, support for versioning or any of those engineer things. All that engineer shit is boring and I can tell myself this laziness is "staying simple" 3. OH SHIT, I was totally right about #1 so this got popular and hav…

Oh yes RFC process always keeps things from having compatibility issues. I definitely never saw any issues with all those XML based standards like SOAP or XSLT.

IETF's RFC process does a pretty good job at keeping things compatible. But I agree W3C specs, especially the SOAP ones are absolutely horrible.

Re: Parsing JSON is a Minefield

#149

Post by taeric »

still_grokking wrote: post #125
taeric wrote: post #82

Earlier quoted context omitted.

The discussion was fun. And seems evenly split, at a quick reading. More, I think it split on how you read it. If you view it as an absolute maxim to excuse poor implementations, it is panned. If you view it as a good faith behavior not to choke on the first mistake, you probably like it. This is akin to grammar police. In life encounters, there is no real place for grammar policing. However, you should try to be gra…

> This is akin to grammar police. In life encounters, there is no real place for grammar policing. However, you should try to be grammatically correct. That's because most humans have feelings. But most machines don't. So that's not comparable.

I meant that grammar policing does little to help the exchange of information. Feelings aside.

Re: Parsing JSON is a Minefield

#150

Post by hyperpape »

drawkbox wrote: post #133
eropple wrote: post #122

Earlier quoted context omitted.

If other people suggesting that, hey, maybe we should actually be able to express a number correctly makes you splutter about "vogons" or whatever, perhaps it is not they who should take a step back. (For this isn't just "massive" numbers, but anything that isn't a float --themselves ranking just after `null` as the worst disaster in current use in general-purpose programming.) Telling people to "just" take actions t…

> If other people suggesting that, hey, maybe we should actually be able to express a number correctly makes you splutter about "vogons" or whatever, perhaps it is not they who should take a step back. I guess what I am saying is JSON was created for simplicity and needs no updates. XML has already been created and other formats like BSON, YAML etc or create a new one that suits more detailed needs. The sole reason t…

Forget XML. You can argue that JSON is better because it’s simpler, and while I’m conflicted, I know I enjoy working with JSON more.

The real question is: with the benefit of hindsight, could you define a better but similarly simple format?

Would an alternative to JSON that specified the supported numeric ranges be less simple? Not really. Would it be better? Yes. The current fact that you can try to represent integers bigger than 2^53, but they lose data, makes no sense except in light of the fact that JSON was defined to work with the quirks of JavaScript.

It's true that different tools are adapted for different uses. But sometimes one tool could have been better without giving up any of what made it useful for its niche.

Post reply on HN