Mostly lost me at "Make all fields in a message required. This makes messages product types." Your constraints on protocols can change over time and context, changing something that is required to something that is optional can cause crashes in prod. Unless somehow the 'optional' design has a way to handle that? You also will have to do custom validation anyways, as your storage format will never be able to enforce a…
If you would continue reading, there is an explanation of how optional would work almost immediately after this quote.
Arguing against using protobuffers
21–30 of 307 posts
Re: Arguing against using protobuffers
#22Good points, but can someone articulate what the best alternative to protobuffers would be in 2018, you know with 'hindsight' etc.?
JSON or graphql. Personally I think the typing, API compatibility and extensive language support make protobuffers currently the winner though, despite the warts
Re: Arguing against using protobuffers
#23Versioning was the biggest disappointment for me. I just want a middleware layer that can handle clients of different versions reliably. Surely everyone has the same problem.
Yes, we do.
But the appropriate answer in most cases is "it's up to the business logic". Trying to solve it at the transport layer is futile.
Re: Arguing against using protobuffers
#24Good points, but can someone articulate what the best alternative to protobuffers would be in 2018, you know with 'hindsight' etc.?
JSON or graphql. Personally I think the typing, API compatibility and extensive language support make protobuffers currently the winner though, despite the warts
Personally, these days I’m taking the approach of just outright avoiding pretty much anything google does. Either because it solves a problem only they have, or they’ll probably just drop it out of the blue one day. Or like Tensorflow, it’s fucking impossible to get your head around because their documentation seems to be written with the goal of being maximally confusing.
Re: Arguing against using protobuffers
#25The utter awkwardness of Protobuf-generated code is particularly problematic. I've had pretty good results with the TypeScript code generator, but the Go generator is just egregiously bad. There's no way to write a client or server that uses the Protobuf structs directly as first-class data types without ending up looking like a ball of spaghetti, at least not if you venture into the realm of oneofs, timestamps or the "well known types" set of types.
I think we're at a juncture where we really desperately need a better, unified way to express schemas and pass type-safe data over the network that flows through these schemas. I'm currently working on a project that involves both gRPC (which uses Protobuf), GraphQL and JSON Schema, with the backends written in Go, and the frontend in TypeScript, and the overlap between all of these is ridiculous. TypeScript is a fresh breath of air that mostly solves the brittleness of JavaScript's type system, and it's absolutely crucial to extend this type-safety all the way to the backend.
The challenge is that no language is the same, and a common schema format ends up targeting a lowest-common denominator. For example, GraphQL errs (in my opinion, wrongly) on the side of simplicity and doesn't support maps, so now you have a whole layer that needs to either custom types (JSON as an escape hatch, basically) or emulate maps using arrays of pairs, neither of which is ideal.
Re: Arguing against using protobuffers
#26Re: Arguing against using protobuffers
#27Re: Arguing against using protobuffers
#28This last is not the case. I've only ever seen it used when people a) already have written serialization code that works with XML/JSON, and b) don't want to write code to serialize to a binary format.
JSON also doesn't let you use objects as object keys, or have any form of polymorphism. It seems like the author wants protobufs to be an RPC system with a modern type system, when in fact it is a data interchange format.
Re: Arguing against using protobuffers
#29Wow, some minor complaints on language formality issues, then discredit the thing as a whole. But in the end, the author probably does not understand PB solves the inter-language data sharing problem, which makes all its complain secondary and inconsequential. It's like complaining a car that drives fast and has great gas efficiency of cannot talk. (Well in 2018, it might be less of a stretch to demand a car that can…
Huh? It's a serialization format. This problem has been solved many times in many different ways.
The author is pointing out that Protobufs were designed poorly, and they didn't have to be.
Re: Arguing against using protobuffers
#30Good points, but can someone articulate what the best alternative to protobuffers would be in 2018, you know with 'hindsight' etc.?