It is a 7 year old article without specifying alternatives to an "already solved problem." So HN, what are the best alternatives available today and why?
Mentioned above: https://github.com/stepchowfun/typical
[deleted]
Protobuffers Are Wrong (2018)
121–130 of 321 posts
Re: Protobuffers Are Wrong (2018)
#122Protocol buffers suck but so does everything else. Name another serialization declaration format that both (a) defines which changes can be make backwards-compatibly, and (b) has a linter that enforces backwards compatible changes. Just with those two criteria you’re down to, like, six formats at most, of which Protocol Buffers is the most widely used. And I know the article says no one uses the backwards compatible…
in the systems I built I didn't bother with backwards compatibility. If you make any change, it's a new message type. For compatibility you can coerce the new message to the old message and dual-publish.
That only works if you control all the clients.
Re: Protobuffers Are Wrong (2018)
#123> Make all fields in a message required.
funnily enough, this line alone reveals the author to be an amateur in the problem space they are writing so confidently about.
Re: Protobuffers Are Wrong (2018)
#124Protocol buffers suck but so does everything else. Name another serialization declaration format that both (a) defines which changes can be make backwards-compatibly, and (b) has a linter that enforces backwards compatible changes. Just with those two criteria you’re down to, like, six formats at most, of which Protocol Buffers is the most widely used. And I know the article says no one uses the backwards compatible…
Flatbuffers satisfies those requirements and doesn’t have varint shenanigans.
What about Cap’n Proto https://capnproto.org/ ? (Don't know much about these things myself, but it's a name that usually comes up in these discussions.)
Re: Protobuffers Are Wrong (2018)
#125Protocol buffers suck but so does everything else. Name another serialization declaration format that both (a) defines which changes can be make backwards-compatibly, and (b) has a linter that enforces backwards compatible changes. Just with those two criteria you’re down to, like, six formats at most, of which Protocol Buffers is the most widely used. And I know the article says no one uses the backwards compatible…
Not widely used but I like Typical's approach https://github.com/stepchowfun/typical > Typical offers a new solution ("asymmetric" fields) to the classic problem of how to safely add or remove fields in record types without breaking compatibility. The concept of asymmetric fields also solves the dual problem of how to preserve compatibility when adding or removing cases in sum types.
This seems interesting. Still not sure if `required` is a good thing to have (for persistent data like log you cannot really guarantee some field's presence without schema versioning baked into the file itself) but for an intermediate wire use cases, this will help.
Re: Protobuffers Are Wrong (2018)
#126Earlier quoted context omitted.
On the other hand, ASN.1 is very expressive and can cover pretty much anything, but Protobuff was created because people thought ASN.1 is too complex. I guess we can't have both.
"Those who cannot remember the past are condemned to repeat it" -- George Santayana
Oh, I remember ASN.1 very well, and I would not want to repeat it again.
Protobufs have lots of problems, but at least they are better than ASN.1!
Re: Protobuffers Are Wrong (2018)
#127Protocol buffers suck but so does everything else. Name another serialization declaration format that both (a) defines which changes can be make backwards-compatibly, and (b) has a linter that enforces backwards compatible changes. Just with those two criteria you’re down to, like, six formats at most, of which Protocol Buffers is the most widely used. And I know the article says no one uses the backwards compatible…
Backwards compatibility is just not an issue in self-describing structures like JSON, Java serialization, and (dating myself) Hessian. You can add fields and you can remove fields. That's enough to allow seamless migrations. It's only positional protocols that have this problem.
You can remove JSON fields at the cost of breaking your clients at runtime that expect those fields. Of course the same can happen with any deserialization libraries, but protobufs at least make it more explicit - and you may also be more easily able to track down consumers using older versions.
Re: Protobuffers Are Wrong (2018)
#128https://news.ycombinator.com/item?id=18190005 Just FYI: an obligatory comment from the protobuf v2 designer. Yeah, protobuf has lots of design mistakes but this article is written by someone who does not understand the problem space. Most of the complexity of serialization comes from implementation compatibility between different timepoints. This significantly limits design space.
>Most of the complexity of serialization comes from implementation compatibility between different timepoints.
The author talks about compatibility a fair bit, specifically the importance of distinguishing a field that wasn't set from one that was intentionally set to a default, and how protobuffs punted on this.
What do you think they don't understand?
Re: Protobuffers Are Wrong (2018)
#129> Make all fields in a message required. funnily enough, this line alone reveals the author to be an amateur in the problem space they are writing so confidently about.
the complaints about the Protobuf type system being not flexible enough are also really funny to read.
fundamentally, the author refuses to contend with the fact that the context in which Protobufs are used -- millions of messages strewn around random databases and files, read and written by software using different versions of libraries -- is NOT the same scenario where you get to design your types once and then EVERYTHING that ever touches those types is forced through a type checker.
again, this betrays a certain degree of amateurishness on the author's part.
Kenton has already provided a good explanation here: https://news.ycombinator.com/item?id=45140590
Re: Protobuffers Are Wrong (2018)
#130Protocol buffers suck but so does everything else. Name another serialization declaration format that both (a) defines which changes can be make backwards-compatibly, and (b) has a linter that enforces backwards compatible changes. Just with those two criteria you’re down to, like, six formats at most, of which Protocol Buffers is the most widely used. And I know the article says no one uses the backwards compatible…
Exactly, I think of protobuffers like I think of Java or Go - at least they weren’t writing it in C++. Dragging your org away from using poorly specified json is often worth these papercuts IMO.
Protobufs are better but not best. Still, by far, the easiest thing to use and the safest is actual APIs. Like, in your application. Interfaces and stuff.
Obviously if your thing HAS to communicate over the network that's one thing, but a lot of applications don't. The distributed system micro service stuff is a choice.
Guys, distributed systems are hard. The extremely low API visibility combined with fragile network calls and unsafe, poorly specified API versioning means your stuff is going to break, and a lot.
Want a version controlled API? Just write in interface in C# or PHP or whatever.