Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Order of the JSON

blog.almaer.com

Post reply on HN
Print view View original

81–90 of 126 posts

Re: The Order of the JSON

#81

Post by cntlzw »

jwilliams wrote: post #46

(I've used both DataPower and COBOL - I've got a healthy respect for robust, long-lived legacy systems). I must admit I was scratching my head on this. The JSON spec might not specify order, but the serialized JSON is ordered by nature. JWT needs it for example (and I'd assume many signature models). Or you might have a caching layer that needs it. Maybe unchecking this causes the legacy backend to get hammered? Ther…

No idea why someone needs ordered key-value-pairs. I had the please of working with a device that relied upon ordered query string parameters. Why? Because they encrypted the values with RC4.

Re: The Order of the JSON

#82

Post by rurban »

AgentOrange1234 wrote: post #19
rurban wrote: post #17

Writers should produce sorted maps, readers need to accept unsorted maps. It's a security issue, not just convenience. With unsorted maps the internal hash seed can be exposed, together with timing information. Another famous omission from the specs.

I’m no security expert. How is “exposing the hash seed” a problem for the vast majority of applications? What timing information would be leaked and why would that be problem? On the other hand, accepting unsorted maps seems like it could introduce covert channels?

> How is “exposing the hash seed” a problem for the vast majority of applications?

They can be DOS'ed.

> What timing information would be leaked and why would that be problem?

You misunderstood. By 1. leaking the order, and 2. by checking the timing of getting certain keys of a map you do have two independent infos to get at the secret seed.

> accepting unsorted maps seems like it could introduce covert channels?

A covert channel might be exposed by the sender, if he uses some non-random but unsorted map order. The receiver needs to accept any order. There's no risk in accepting unsorted maps. The only risk at the receiver side is another famous omission from the spec: How to deal with duplicate keys. Accept (overwrite or drop) or reject? All 3 cases can be seen in the wild.

Re: The Order of the JSON

#83

Post by clusmore »

jwilliams wrote: post #46

(I've used both DataPower and COBOL - I've got a healthy respect for robust, long-lived legacy systems). I must admit I was scratching my head on this. The JSON spec might not specify order, but the serialized JSON is ordered by nature. JWT needs it for example (and I'd assume many signature models). Or you might have a caching layer that needs it. Maybe unchecking this causes the legacy backend to get hammered? Ther…

I can't help but think of the story of Chesterton's Fence [1].

>There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, 'I don't see the use of this; let us clear it away.' To which the more intelligent type of reformer will do well to answer: 'If you don't see the use of it, I certainly won't let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.'

[1]: https://en.m.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fen...

Re: The Order of the JSON

#84

Post by johnchristopher »

Ididntdothis wrote: post #5

This sounds very familiar. A lot of companies are full of people who have no curiosity and no ability to think for themselves. I have seen it multiple times where someone claimed a change is impossible or takes insane effort. Then you have someone competent look at it and you have a solution in an hour. I think stuff likes this is the real price of not hiring really good people.

I have been `the competent guy` this week. It took me two years to realize how deep the willingness to do nothing is for the IT guys here. I arrived as usual in the morning last week. An employee told me the notification system was broken and wouldn't be fixed before next year. I told her I would implement a fix before the end of the day. I got it done. The IT guy still has his public servant job and I am told to write a note to justify the implementation of the fix (I am not a public servant and not a contractor, I work for a private public LLC that is managed by a regional authority).

Re: The Order of the JSON

#85

Post by smacktoward »

Someone wrote: post #49

Next installment: how flipping an innocuous switch introduced a subtle bug that was silent for years and cost us millions. You can’t just flip that switch, run ”a client to post the JSON to that instance” , see that test “worked just fine” , and call it a day. The POST might just store it, for later processing to wreak havoc (say by ignoring a value that isn’t in the expected place in the JSON), or only rarely seen J…

Also known as Chesterton’s Fence: https://en.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence

Re: The Order of the JSON

#86

Post by [deleted] »

jeremy_wiebe wrote: post #25
duxup wrote: post #14

Earlier quoted context omitted.

Write some software that reordered it probably.

Yep. Probably IBM Data Transform Services™️ installed in the cloud plus a custom plugin written to do the ordering. That’s a few folks to deploy and configure the server. A few more for the engineering. Oh and there’s probably a support contract to support and maintain this new “solution”.

[deleted]

Re: The Order of the JSON

#87

Post by dtech »

Someone wrote: post #49

Next installment: how flipping an innocuous switch introduced a subtle bug that was silent for years and cost us millions. You can’t just flip that switch, run ”a client to post the JSON to that instance” , see that test “worked just fine” , and call it a day. The POST might just store it, for later processing to wreak havoc (say by ignoring a value that isn’t in the expected place in the JSON), or only rarely seen J…

How do you justify ever changing anything with that attitude?

This change made parsing more lenient which is generally considered fine [1] and made the software actually follow the spec, since it specifies object entries are unordered.

[1] https://en.wikipedia.org/wiki/Robustness_principle

Re: The Order of the JSON

#88

Post by dtech »

lilyball wrote: post #27

The tweet at the top of the article says > This so far out of the spec it makes my ankles hurt. This is not in fact out of spec. The JSON spec does not define semantics here but instead quite explicitly leaves it up to the JSON processor and data interchange spec for what to do about ordering of objects.

> This is not in fact out of spec. The JSON spec does not define semantics [...] for what to do about ordering of objects.

What? It is literally on the first page of json.org and section 1 of RFC 7159 [1]

> An object is an unordered set of name/value pairs.

> An object is an unordered collection of zero or more name/value pairs

[1] https://tools.ietf.org/html/rfc7159#section-1

Re: The Order of the JSON

#89

Post by tantalor »

dtech wrote: post #87
Someone wrote: post #49

Next installment: how flipping an innocuous switch introduced a subtle bug that was silent for years and cost us millions. You can’t just flip that switch, run ”a client to post the JSON to that instance” , see that test “worked just fine” , and call it a day. The POST might just store it, for later processing to wreak havoc (say by ignoring a value that isn’t in the expected place in the JSON), or only rarely seen J…

How do you justify ever changing anything with that attitude? This change made parsing more lenient which is generally considered fine [1] and made the software actually follow the spec, since it specifies object entries are unordered. [1] https://en.wikipedia.org/wiki/Robustness_principle

> How do you justify ever changing anything

By rolling out the change over a period of time to a small sample of users and comparing the experimental effect to a control sample.

Re: The Order of the JSON

#90

Post by tantalor »

crazygringo wrote: post #10
hyperpallium wrote: post #8

Earlier quoted context omitted.

OK. I totally prefer ordered JSON, because it is so much easier to eyeball - to visually compare JSON with different ordered keys is quite a lot more difficult (O() complexity?) than if they are in the same order. It also enables diff to help see where the differences are (diagnosis, not just binary identical or not). And, in fact, I do use ordered JSON for comparison in testing, as you describe. However ... comparis…

+1 for all reasons above for ordered JSON, highly convenient in practice. And if it doesn't impact performance significantly, these are all pretty good reasons for JSON outputters to default to sorting objects deterministically by keys, or at least to provide a flag to do so. (Even if there's no canonical sort order between JSON libraries, all that matters is it's deterministic for each library.) BUT... I can't imagi…

> any scenario where you'd want to validate...

Because it's a precondition for something else down the line (a dependency)

Post reply on HN