Good points, but can someone articulate what the best alternative to protobuffers would be in 2018, you know with 'hindsight' etc.?
Arguing against using protobuffers
41–50 of 307 posts
Re: Arguing against using protobuffers
#42Versioning was the biggest disappointment for me. I just want a middleware layer that can handle clients of different versions reliably. Surely everyone has the same problem.
it doesn't mean that there is a single solution that will satisfy everyone
Re: Arguing against using protobuffers
#43> Option 1 is clearly the "right" solution, but its untenable with protobuffers. The language isn't powerful enough to encode types that can perform double-duty as both wire and application formats. Which means you'd need to write a completely separate datatype, evolve it synchronously with the protobuffer, and explicitly write serialization code between the two. Seeing as most people seem to use protobuffers in orde…
The only restriction I can think of that JSON imposes (in terms of composition) is that keys must be strings. And I don't necessarily think JSON is a great serialization format either.
Re: Arguing against using protobuffers
#44Re: Arguing against using protobuffers
#45Good points, but can someone articulate what the best alternative to protobuffers would be in 2018, you know with 'hindsight' etc.?
Not sure if the best, but maybe has different tradeoffs
Re: Arguing against using protobuffers
#46It's abundantly clear why this author lasted only a year at Google. Aside from using the non-idiomatic "protobuffers" ... "nothing wants to inspect only some bits of a message and then forward it on unchanged" In my experience it is extremely useful to partially parse a protobuf, or to not parse it at all and simply modify it by appending to it. Also useful is the ability to define a message type that is isomorphic o…
The argument presented is that protobuf does not attempt to even replicate the best practices we already have in terms of data representation. It makes sense why languages like C are constrained in their data representations -- they want to be close to the machine. However, a language meant literally to specify data really ought to be able to handle the basic competencies of its field (like sum and product types and polymorphism) with ease.
There is no excuse in 2018 for a data language without polymorphism and without product and co-product types. Compatibility with legacy or substandard languages is not an excuse for poor design. Compatibility with needed legacy languages should be the thing that's tacked on, not basic functionality.
Re: Arguing against using protobuffers
#47Though I dislike the hyperbolic tone and personal attacks, the author isn't entirely wrong. There are many design choices in Protocol Buffers that seem directly related to the scale and complexity at which Google operates, and which sacrifice safety, clarity and language integration. The utter awkwardness of Protobuf-generated code is particularly problematic. I've had pretty good results with the TypeScript code gen…
Write your own generator then?
Re: Arguing against using protobuffers
#48 struct mytype s;
s.field1 = something;
s.field2 = something else;
send(socket, &s, sizeof(s), 0)
or, using a language with a good type system like Haskell data MyData = MyData Int Int deriving Generic
instance Storable MyData
alloca $ \buf -> do
let d = MyData field1 field2
poke buf d
send socket buf (sizeof d)Re: Arguing against using protobuffers
#49> Fields with scalar types are always present. Even if you don't set them. Did I mention that (at least in proto3) all protobuffers can be zero-initialized with absolutely no data in them? I don't this complaint. You have to initialize data with something . proto2 had to "solve" this problem in C/C++/Go by making everything a pointer. How is dealing with null the "sane" case against zero-initialization?
The class layout for proto3 is the same (surprise!) but the presence vector is ignored.
Re: Arguing against using protobuffers
#50> Fields with scalar types are always present. Even if you don't set them. Did I mention that (at least in proto3) all protobuffers can be zero-initialized with absolutely no data in them? I don't this complaint. You have to initialize data with something . proto2 had to "solve" this problem in C/C++/Go by making everything a pointer. How is dealing with null the "sane" case against zero-initialization?
Go has its own philosophy about zero values which is controversial, but at least with pointers you do get to express optional values. And it's not like you can't work around the ergonomic awkwardness that arises:
type Post struct {
Title *string
}
func (p *Post) GetTitle() (string, bool) {
if p.Title == nil {
return "", false
}
return *p.Title, true
}
func (p *Post) SetTitle(s string) {
p.Title = &s
}
Hardly elegant, but this is Go.Go also run into the same issue when encoding and decoding JSON. Most languages do distinguish between empty string and a missing string, but not Go, which makes it hard to validate anything. I wrote a code generator for JSON Schema [1] recently, which applies validations while deserializing, and has to resort to a rather low-tech trick to do so:
type Post struct {
Title string `json:"title"`
}
func (v *Post) UnmarshalJSON(b []byte) error {
var raw map[string]interface{}{
if err := json.Unmarshal(b, &raw); err != nil {
return err
}
if _, ok := raw["title"]; !ok {
return errors.New("field title: must be set")
}
type plain Post
var p plain
if err := json.Unmarshal(b, &p); err != nil {
return err
}
*v = Post(p)
return nil
}
(Clearly there are slightly more refined and faster ways to do this, but not that easily without importing third-party code, which I wanted to avoid.)C and Go are in the minority here. This doesn't come up in languages like Rust, Swift, TypeScript, Nim, Haskell, OCaml, C#, F#, or Java >= 8, all of which have some sort of optional support (algebraic data types or built-in). For example, TypeScript:
interface Post {
title?: string
}
Or Rust: struct Post {
title: Option
}
[1] https://github.com/atombender/go-jsonschema