Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Parsing malformed JSON

peteris.rocks

Post reply on HN
Print view View original

51–57 of 57 posts

Re: Parsing malformed JSON

#51

Post by brassic »

hueving wrote: post #50
mwkaufma wrote: post #41

Or, how I made my service a DDoS target. It's not just the extra compute, it's the lack of a formal specification. If different services applied this kind of ad hoc "postel's principle" they may parse the malformed markup differently, and end up introducing downstream inconsistencies.

Or even vulnerabilities. Imagine a scenario where a parser for an authentication engine reads a different value for a given key than the value the authorization logic reads.

This isn't theoretical, I've seen it with HTTP, HTML and elsewhere. Any time two pieces of software disagree on how to parse a chunk of data, especially if one of them is supposed to be doing some sort of security check, you should expect to find a vulnerability lurking.

I don't know if there's a name for this class of problem. I'd be interested to know.

Re: Parsing malformed JSON

#52

Post by ekiara »

Wouldn't a better option be an error log? you reply to the client that "I can accept 398,500 of your 400,000 submitted records, attached are the records that do not conform to the expected template. Choose either to (1) submit only the validated records and discard the malformed ones or (2) reformat the malformed records and resubmit the entire batch"

Re: Parsing malformed JSON

#53

Post by ezrast »

ludamad wrote: post #48
mertd wrote: post #43

Earlier quoted context omitted.

I think nom-nom is trying to imply that if you're passing GBs of Json around, "human readability" isn't probably a concern. Therefore you could go for an efficient binary format.

JSON isn't just about human readability, it's about being a 'good enough' standard for data exchange. What binary format would you use that people could parse as reliably as JSON?

In case your question wasn't rhetorical, I believe MessagePack is the leading schema-less binary serialization format (which does not contradict your point as it is still less ubiquitous than JSON).

Re: Parsing malformed JSON

#54

Post by nommm-nommm »

bootload wrote: post #46
nommm-nommm wrote: post #17

Why would a JSON file be GBs in size? I think that's the more interesting question.

"Why would a JSON file be GBs in size?" Maps. [0] Geo-cordinate data can consist of tens of thousands of data points. For example, think of a two dimensional space with a co-ordinate grid at regular intervals representing a 20km x 20km city. Then imagine creating an outline of a city road network. Each point a LAT/LON co-ordinate. Then imagine placing thematic data such as known traffic hot spots. Lots of data. [0] T…

Thank you! I was really racking my brain trying to think of a use case that produced that much JSON.

Re: Parsing malformed JSON

#55

Post by wccrawford »

k__ wrote: post #6
wccrawford wrote: post #5

It's a neat trick, but not something I'd deploy into production. If I have to try to guess at what the customer is sending me, I'm not going to apply it to their account. In an emergency, I might hand-edit it and make it right, but I'd absolutely insist that further files be in the correct format.

Isn't this used mainly in editors that want to provide some hints even for JSON you didn't finished yet.

That's a legit use for it, sure. But when a non-techie sees something like this, they immediately think of all the hassle they can save a customer that is having trouble making valid JSON. "We'll just parse it for them!" They completely ignore that it's not possible to know for sure what the customer really wanted, and it's the start of a lot of headaches.

Re: Parsing malformed JSON

#56

Post by treve »

kr0 wrote: post #36
treve wrote: post #25

Earlier quoted context omitted.

I'm doing this with WebDAV too. When I come across a bug that's clearly an implementation problem I weigh how prevalent the software is, how likely they will be able to fix it and if possible I add a user-agent specific workaround so new clients can't rely on the same bug with my server.

But then we add the IE-nightmare of using an accepted user-agent in a new product to workaround cases like this

That nightmare had to do with misbehaving servers. IE had to advertise as Mozilla so servers would serve the better response.

In this case it would be possible for a client to fake a UA, but it's more likely that they weren't aware they were doing things incorrectly and correct the behavior rather than opting in to mimicing a different UA to get the server to behave in a non-standard way.

I haven't seen this happen, and this is one of the most popular DAV implementations. I have seen people fix broken implementations as I've slowly been making the server more strict over the last 10 years.

Re: Parsing malformed JSON

#57

Post by ape4 »

JadeNB wrote: post #39
ape4 wrote: post #26

JSON should have a nicer way of dealing with double quotes in data. That would avoid many encoding mistakes.

> JSON should have a nicer way of dealing with double quotes in data. That would avoid many encoding mistakes. So you update the standard to this nicer way of dealing with double quotes, and now people forget to indicate whether they're using the nice new way or the ugly old way, or they mix the two approaches ….

It would have to be phased in ... like html5 or any browser improvement.
Post reply on HN