Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Order of the JSON

blog.almaer.com

Post reply on HN
Print view View original

61–70 of 126 posts

Re: The Order of the JSON

#61

Post by dagw »

Someone wrote: post #49

Next installment: how flipping an innocuous switch introduced a subtle bug that was silent for years and cost us millions. You can’t just flip that switch, run ”a client to post the JSON to that instance” , see that test “worked just fine” , and call it a day. The POST might just store it, for later processing to wreak havoc (say by ignoring a value that isn’t in the expected place in the JSON), or only rarely seen J…

Yea, that was my big question as well. Was the switch just for that one end point or did he accidentally reconfigure how all the API endpoints in the entire company worked.

Re: The Order of the JSON

#62

Post by moby_click »

jwilliams wrote: post #46

(I've used both DataPower and COBOL - I've got a healthy respect for robust, long-lived legacy systems). I must admit I was scratching my head on this. The JSON spec might not specify order, but the serialized JSON is ordered by nature. JWT needs it for example (and I'd assume many signature models). Or you might have a caching layer that needs it. Maybe unchecking this causes the legacy backend to get hammered? Ther…

If you require serialized JSON in a specific order, you are not requiring JSON, but some dialect of JSON and that should be clearly communicated. If, for some reason, a client needed a special serialization, i'd ask them for a spec, because the JSON spec does not apply anymore, and be done with it.

Re: The Order of the JSON

#63

Post by mirimir »

> Much of the system was so old that it was hard to find anyone who knew how it actually worked, and it’s maintenance had been outsourced to some of the typical IT outsourcing companies of the time.

Some years ago, I worked on an antitrust case involving a firm that had outsourced relevant systems. Multiple times, to different IT firms. And there was literally nobody left who knew how they worked.

After considerable negotiation, they agreed to provide documentation. And what that ended up being was a report by IT company 2 about their understanding of what IT company 1 had done with the firm's systems. Because, I gather, IT company 1 had evaporated.

And yes, the core of it was COBOL.

Re: The Order of the JSON

#64

Post by josephg »

al_form2000 wrote: post #54

"Why would you care what order we were sending the name value pairs for this?" The article makes it sound like it's somehow IBM's fault. However, questions about preserving order in hashes/associative arrays/maps/whatchamacallit span at least thirty years. Some applications of the format even demand it (e.g. ansible uses YAML - a superset of JSON - or even straight JSON for it playbooks; emitting them out of order is…

I hear that. But also, JSON does not have a canonical serialisation. Other than object key ordering, strings have multiple equivalent formats (direct Unicode characters, \u00xx, \xXX, etc). Numbers have exponential form (1e2 is the same as 100). And white space is ignored - [2] and [ 2 ] and [2\n] are all equivalent.

If your application expects JSON input to conform to one particular set of decision points here, that’s fine and useful but you’re not using JSON anymore. You’re using a JSON-compatible subset. Document that new thing you’ve made and stop calling it JSON; because it’s not.

Re: The Order of the JSON

#65

Post by atemerev »

blattimwind wrote: post #47
jwilliams wrote: post #46

(I've used both DataPower and COBOL - I've got a healthy respect for robust, long-lived legacy systems). I must admit I was scratching my head on this. The JSON spec might not specify order, but the serialized JSON is ordered by nature. JWT needs it for example (and I'd assume many signature models). Or you might have a caching layer that needs it. Maybe unchecking this causes the legacy backend to get hammered? Ther…

> JWT needs it for example (and I'd assume many signature models) You almost never need canonical representations for signing things. I would even say that if you need a canonical representation to sign your things, then that is a design smell of your cryptographic protocol.

Could you please care to explain more? What is the "design smell", and what is the alternative solution to canonicalization?

Re: The Order of the JSON

#66

Post by dagw »

jrochkind1 wrote: post #12

How were they going to charge him for 9 FTEs for 6 months to uncheck a checkbox?

Perhaps they didn't know about the checkbox and thought they had to write a custom solution. It could also be that they knew about that really subtle bug that is caused by un-checking that checkbox and thus understood why that didn't actually solve the problem long term if though it looked like a good solution at the time.

Or perhaps they're so used to getting away with quoting 6 month for a 6 hour job, it never crossed their mind not to.

Re: The Order of the JSON

#67

Post by jwilliams »

moby_click wrote: post #62
jwilliams wrote: post #46

(I've used both DataPower and COBOL - I've got a healthy respect for robust, long-lived legacy systems). I must admit I was scratching my head on this. The JSON spec might not specify order, but the serialized JSON is ordered by nature. JWT needs it for example (and I'd assume many signature models). Or you might have a caching layer that needs it. Maybe unchecking this causes the legacy backend to get hammered? Ther…

If you require serialized JSON in a specific order, you are not requiring JSON, but some dialect of JSON and that should be clearly communicated. If, for some reason, a client needed a special serialization, i'd ask them for a spec, because the JSON spec does not apply anymore, and be done with it.

I think my argument is a bit different -- sure, this is an interface (maybe to a COBOL copybook in this case?). And yes, it's not "JSON".

Agree it's a leaky abstraction. Equally agree it should be documented. Often legacy systems have weirdness after seeing decades of edge-cases. Weirdness that makes them robust in all sorts of unlikely ways. Equally makes them poorly documented, leaky, opaque, and frustrating.

But I certainly have a lot of pause with the sentiment of "some idiot checked the JSON order checkbox on DataPower". My first thought is instead "I wonder why someone thought this was necessary."

Re: The Order of the JSON

#68

Post by Piskvorrr »

dusted wrote: post #48
Ididntdothis wrote: post #5

This sounds very familiar. A lot of companies are full of people who have no curiosity and no ability to think for themselves. I have seen it multiple times where someone claimed a change is impossible or takes insane effort. Then you have someone competent look at it and you have a solution in an hour. I think stuff likes this is the real price of not hiring really good people.

Unfortunately, there simply are not enough "really good" people around, so they'll have to make do with the likes of me ;-)

Oh, don't worry. If I'm capable of improving, so are you ;)

Re: The Order of the JSON

#69

Post by jwilliams »

blattimwind wrote: post #47
jwilliams wrote: post #46

(I've used both DataPower and COBOL - I've got a healthy respect for robust, long-lived legacy systems). I must admit I was scratching my head on this. The JSON spec might not specify order, but the serialized JSON is ordered by nature. JWT needs it for example (and I'd assume many signature models). Or you might have a caching layer that needs it. Maybe unchecking this causes the legacy backend to get hammered? Ther…

> JWT needs it for example (and I'd assume many signature models) You almost never need canonical representations for signing things. I would even say that if you need a canonical representation to sign your things, then that is a design smell of your cryptographic protocol.

I didn't mention canonicalization. My point is that serialized JSON is ordered - which I think is exactly the same property you're referring to.

Re: The Order of the JSON

#70

Post by Piskvorrr »

al_form2000 wrote: post #56
shortercode wrote: post #35

Earlier quoted context omitted.

If the objects are not guaranteed to be in order, I would always consider them unordered. You could if both the producer and consumer are ordered, but as the OP noted this can cause problems later on and for no good reason. I'd use an array of key-value tuples if I wanted it to be ordered.

Uh, "no good reason" in this case means "I unchecked the box and nothing bad happened right away" which amounts to shallow analysis, to put it gently.

That was a twist ending, indeed. I was expecting a resolution, not "I gave it a good whack and guess what, that fixed it."
Post reply on HN