Arguing against using protobuffers

Post by taeric » Wed, Oct 10, 2018, 10:37 PM UTC

hardwaresofton wrote: post #67

atombender wrote: post #25

Though I dislike the hyperbolic tone and personal attacks, the author isn't entirely wrong. There are many design choices in Protocol Buffers that seem directly related to the scale and complexity at which Google operates, and which sacrifice safety, clarity and language integration. The utter awkwardness of Protobuf-generated code is particularly problematic. I've had pretty good results with the TypeScript code gen…

I'd like to humbly suggest that we use JSON please, in particular: JSON + JSONSchema[0] +/- JSON Hyperschema[1] +/- JSON LD[2] It's a bit to learn but I promise you, it's worth it. The technologies are not redundant (jsonschema spec is for validation, hyperschema spec is for specifying how you interact, and LD is for semantics like language and more). If you take a few hours, read all 3 specs, you're almost guarantee…

Post by tathougies » Wed, Oct 10, 2018, 10:38 PM UTC

amluto wrote: post #63

tathougies wrote: post #48

Easiest way to send data over the wire. struct mytype s; s.field1 = something; s.field2 = something else; send(socket, &s, sizeof(s), 0) or, using a language with a good type system like Haskell data MyData = MyData Int Int deriving Generic instance Storable MyData alloca $ \buf -> do let d = MyData field1 field2 poke buf d send socket buf (sizeof d)

This may be easy, but it’s wrong . Endianness issues are just the start. Information leaks due to padding are a big deal. And it straight up doesn’t work if nontrivial data structures are involved.

Post by patrec » Wed, Oct 10, 2018, 10:38 PM UTC

danharaj wrote: post #13

jeremyjh wrote: post #7

Earlier quoted context omitted.

Which of the problems listed in the article are solved by it?

Parameterization of types, off the top of my head.

Post by tathougies » Wed, Oct 10, 2018, 10:38 PM UTC

ericpauley wrote: post #62

tathougies wrote: post #48

Easiest way to send data over the wire. struct mytype s; s.field1 = something; s.field2 = something else; send(socket, &s, sizeof(s), 0) or, using a language with a good type system like Haskell data MyData = MyData Int Int deriving Generic instance Storable MyData alloca $ \buf -> do let d = MyData field1 field2 poke buf d send socket buf (sizeof d)

C doesn't make struct memory layout guarantees, endian guarantees, or size guarantees in some cases, so unless you're running the exact same binary this is a very poor serialization technique.

Post by Vendan » Wed, Oct 10, 2018, 10:40 PM UTC

atombender wrote: post #50

nemothekid wrote: post #17

> Fields with scalar types are always present. Even if you don't set them. Did I mention that (at least in proto3) all protobuffers can be zero-initialized with absolutely no data in them? I don't this complaint. You have to initialize data with something . proto2 had to "solve" this problem in C/C++/Go by making everything a pointer. How is dealing with null the "sane" case against zero-initialization?

To be fair, in C++ you could use std::optional or something else that's type-safe. Go has its own philosophy about zero values which is controversial, but at least with pointers you do get to express optional values. And it's not like you can't work around the ergonomic awkwardness that arises: type Post struct { Title *string } func (p *Post) GetTitle() (string, bool) { if p.Title == nil { return "", false } return…

    type Post struct {
    	Title string `json:"title"`
    }

    func (v *Post) UnmarshalJSON(b []byte) error {
    	dec := json.NewDecoder(bytes.NewReader(b))
    	required := map[string]struct{}{
    		"title": struct{}{},
    	}
    	tok, err := dec.Token()
    	if err != nil {
    		return err
    	}
    	if d, ok := tok.(json.Delim); !ok || d != '{' {
    		return errors.New("Expected object")
    	}
    	for {
    		tok, err := dec.Token()
    		if err != nil {
    			return err
    		}
    		if d, ok := tok.(json.Delim); ok && d == '}' {
    			break
    		}
    		switch tok {
    		case "title":
    			delete(required, "title")
    			err := dec.Decode(&v.Title)
    			if err != nil {
    				return err
    			}
    		default:
    			(*v) = Post{}
    			return errors.New(fmt.Sprintf("Unexpected field %s", tok))
    		}
    	}
    
    	if len(required) > 0 {
    		(*v) = Post{}
    		return errors.New(fmt.Sprintf("Missing %v required fields", len(required)))
    	}
    	return nil
    }