Live data from Hacker News

A new experimental Go API for JSON

go.dev

21–30 of 110 posts

Re: A new experimental Go API for JSON

#21
post #15

> Over time, packages evolve with the needs of their users, and encoding/json is no exception No, it's an exception. It was badly designed from the start - it's not just that people's json needs (which hardly changed) outgrew it.

A bad design doesn't invalidate the sentence you have quoted. Over time, it became evident that the JSON package didn't meet the needs of its users, and the package has evolved as a result. The size of the evolution doesn't matter.

It's true that packages (generally) evolve with the needs of their users.

It's also true that a json IO built-in lib typically wouldn't be so poorly designed in the first release of a language, that it would immediately be in need of maintenance.

Re: A new experimental Go API for JSON

#22

Earlier quoted context omitted.

A bad design doesn't invalidate the sentence you have quoted. Over time, it became evident that the JSON package didn't meet the needs of its users, and the package has evolved as a result. The size of the evolution doesn't matter.

It's true that packages (generally) evolve with the needs of their users. It's also true that a json IO built-in lib typically wouldn't be so poorly designed in the first release of a language, that it would immediately be in need of maintenance.

> immediately

JSON library released with Go 1, in 2012. This makes the library 13 years old [0].

If that's immediate, I'm fine with that kind of immediate.

[0]: https://pkg.go.dev/encoding/json@go1

Re: A new experimental Go API for JSON

#23

Earlier quoted context omitted.

It's true that packages (generally) evolve with the needs of their users. It's also true that a json IO built-in lib typically wouldn't be so poorly designed in the first release of a language, that it would immediately be in need of maintenance.

> immediately JSON library released with Go 1, in 2012. This makes the library 13 years old [0]. If that's immediate , I'm fine with that kind of immediate. [0]: https://pkg.go.dev/encoding/json@go1

In need of maintenance and having received maintenance are two different things

Re: A new experimental Go API for JSON

#24
post #15

> Over time, packages evolve with the needs of their users, and encoding/json is no exception No, it's an exception. It was badly designed from the start - it's not just that people's json needs (which hardly changed) outgrew it.

A bad design doesn't invalidate the sentence you have quoted. Over time, it became evident that the JSON package didn't meet the needs of its users, and the package has evolved as a result. The size of the evolution doesn't matter.

"Becoming evident it doesn't meet the needs of its users" is not the same as "packages evolve with the needs of their users".

The latter is a weasely way to put the blame on changing needs - as if initially it was fine, but user needs grew and it's not covering them anymore. Truth is, user needs are the same, we havent had any magical change in JSON use patterns over the last 10 years. The design was just flawed to begin with.

I'd argue it didn't "become evident over time" either. It was evident on day one, and many people pointed it out 10 and 13 years ago.

Re: A new experimental Go API for JSON

#25

Earlier quoted context omitted.

It's true that packages (generally) evolve with the needs of their users. It's also true that a json IO built-in lib typically wouldn't be so poorly designed in the first release of a language, that it would immediately be in need of maintenance.

> immediately JSON library released with Go 1, in 2012. This makes the library 13 years old [0]. If that's immediate , I'm fine with that kind of immediate. [0]: https://pkg.go.dev/encoding/json@go1

"immediately be in need of maintenance" means it needed this update 13 years ago.

Re: A new experimental Go API for JSON

#26
post #6

Benchmark Analysis: Sonic vs Standard JSON vs JSON v2 in Go https://github.com/centralci/go-benchmarks/tree/b647c45272c7...

Those numbers look similar to goccy. I used to use it in the past, even Kubernetes uses it as direct dependency, but the amount of issues have been stockpiling for quite some time so I no longer trust it. So it seems both are operating at the edge of Go's capabilities. Personally, I think JSON should be in Go's core and highly optimised simd c code and not in the Go's std library as standard Go code. As JSON is such…

Agreed. goccy has better performance most times but absolutely appalling worst-case performance which renders it unacceptable for many use cases - in my case even with trusted input it took effectively eternity to decode it. It's literally a quadratic worst case, what's the point of having a bunch of super clever optimisations if the big-O performance is that bad.

Sonic may be different but I'm feeling once bitten twice shy on "faster" JSON parsers at this point. A highly optimised SIMD version might be nice but the stdlib json package needs to work for everything out there, not just the cases the author decided to test on, and I'd be a lot more nervous about something like that being sufficiently well tested given the extra complexity.

Re: A new experimental Go API for JSON

#27

Could somebody give a high level overview of this for me, as not a godev? It looks like Go JSON lib has support to encode native go structures in JSON, which is cool, but maybe it was bad, which is not as cool. Do I have that right?

Go already has a JSON parser and serializer. It kind of resembles the JS api where you push some objects into JSON.stringify and it serializes them. Or you push some string and get an object (or string etc) from JSON.parse.

The types themselves have a way to customize their own JSON conversion code. You could have a struct serialize itself to a string, an array, do weird gymnastics, whatever. The JSON module calls these custom implementations when available.

The current way of doing it is shit though. If you want to customize serialization, you need to return a json string basically. Then the serializer has to check if you actually managed to return something sane. You also have no idea if there were some JSON options. Maybe there is an indentation setting or whatever. No, you return a byte array.

Deserialization is also shit because a) again, no options. b) the parser has to send you a byte array to parse. Hey, I have this JSON string, parse it. If that JSON string is 100MB long, too bad, it has to be read completely and allocated again for you to work on because you can only accept a byte array to parse.

New API fixes these. They provide a Decoder or Encoder to you. These carry any options from top. And they also can stream data. So you can serialize your 10GB array value by value while the underlying writer writes it into disk for example. Instead of allocating all on memory first, as the older API forces you to.

There are other improvements too but the post mainly focuses on these so thats what I got from it (I havent tried the new api btw, this is all from the post so maybe I’m wrong on some points)

Re: A new experimental Go API for JSON

#29
post #15

> Over time, packages evolve with the needs of their users, and encoding/json is no exception No, it's an exception. It was badly designed from the start - it's not just that people's json needs (which hardly changed) outgrew it.

Well mostly I have seem people learn shortcomings of software by using or creating it and come up with new version when possible. In your case it seems v1 are perfect each time.

Re: A new experimental Go API for JSON

#30
post #28

> Since encoding/json marshals a nil slice or map as a JSON null How did that make it into the v1 design?

I had a back and forth with someone who really didn't want to change that behavior and their reasoning was that since you can create and provide an empty map or slice.. having the marshaler do that for you, and then also needing a way to disable that behavior, was unnecessary complexity.
Post reply on HN