Earlier quoted context omitted.
Because they would had to add a `/v2` suffix to the import path and seems like they didn't wanted to add that. The question is, why is this not released as v1.0.0 then? Still trying to understand the reasoning for this.
Reasoning went something like this: We could tag the new API v2: + Makes the "v1" and "v2" distinction very clear in the import path. - Confusing: google.golang.org/protobuf@v1 doesn't exist, but v2 does. - In ten years, hopefully nobody cares about the old github.com/golang/protobuf and the confusion is gone. We could tag the new API v1: - Less visually distinct in the import path. + Seems to make sense for the firs…
A new Go API for Protocol Buffers
31–40 of 100 posts
Re: A new Go API for Protocol Buffers
#32One perennial annoyance is how awkwardly some Protobuf stuff ends up being represented in Go. For example, "oneof" types:
message Event {
oneof payload {
Create create = 1;
Delete delete = 2;
}
}
message Create {
string id = 1;
}
message Delete {
string version = 1;
}
You get these messy types (I've elided most of the yucky marshaling-related stuff): type Event struct {
Payload isEvent_Payload `protobuf_oneof:"item"`
}
type isEvent_Payload interface {
isEvent_Payload()
MarshalTo([]byte) (int, error)
Size() int
}
type Event_Create struct {
Create *Create `protobuf:"bytes,1,opt,name=create,proto3,oneof"`
}
type Event_Delete struct {
Delete *Delete `protobuf:"bytes,1,opt,name=delete,proto3,oneof"`
}
There are multiple problems here. One: Why the extra struct? For example, to create a single event, you have to do: Event{
Payload: &Event_Create{
Create: &Create{ID: "123"},
},
},
...instead of just: Event{
Payload: &Create{ID: "123"},
},
Secondly, isEvent_Payload is private! So given a Create or an Delete, you can't store it in a single variable: func nextEvent() isEvent_Payload { //
The reason for this appears to be so that the generated serialization can be very dumb and not rely on type switches. But I don't buy that this is how it has to be.There are other issues. Overall, the whole package seems designed from the bottom up for machines (mumbles possibly also by machines), not humans.The upshot is that using Protobuf types as "first-class" types — meaning the types you actually use internally in the meat of your app, as opposed to in the controller glue that lives in your API and mediates between the API and the internals — feels super messy.
As an aside, anyone know what issues this paragraph refers to?
The google.golang.org/protobuf/encoding/protojson package
converts protocol buffer messages to and from JSON using the
canonical JSON mapping, and fixes a number of issues with the
old jsonpb package that were difficult to change without
causing problems for existing users.
Did they finally fix the zero value problem with jsonpb.go [1]?Re: A new Go API for Protocol Buffers
#33I thought the general consensus on protocol buffers was "Meh"?
I think for large organizations they probably make sense. They're potentially a lot less data per request than JSON. For the average app? Overkill, sure.
Re: A new Go API for Protocol Buffers
#34Earlier quoted context omitted.
Because they would had to add a `/v2` suffix to the import path and seems like they didn't wanted to add that. The question is, why is this not released as v1.0.0 then? Still trying to understand the reasoning for this.
Reasoning went something like this: We could tag the new API v2: + Makes the "v1" and "v2" distinction very clear in the import path. - Confusing: google.golang.org/protobuf@v1 doesn't exist, but v2 does. - In ten years, hopefully nobody cares about the old github.com/golang/protobuf and the confusion is gone. We could tag the new API v1: - Less visually distinct in the import path. + Seems to make sense for the firs…
Also:
> In ten years, hopefully nobody cares about the old github.com/golang/protobuf and the confusion is gone.
is the corollary that this change will cause 10 years of confusion?
Re: A new Go API for Protocol Buffers
#35On the Rust side, macros can[1] transform .proto files into data structure definitions at compile time, and instances of the structs passed around have the benefit of strong typing, code completion in editors, lints and tests can easily validate properties of protobuf objects and so on. And all without adding more codegen steps which is nice - the macros can replace a build system having to do codegen as a separate step.
How often do you need to mutate any protocol buffer and remove (redact as they say) fields generating a completely invalid protocol buffer object? Will anything downstream match the ".proto" of an object that's missing a bunch of required fields? Wouldn't it be easier to just all of the sensitive data into an optional "sensitive_data" field that's defined by another .proto, and strip that?
[1] - https://github.com/danburkert/prost#generated-code-example
Re: A new Go API for Protocol Buffers
#36Earlier quoted context omitted.
I think for large organizations they probably make sense. They're potentially a lot less data per request than JSON. For the average app? Overkill, sure.
Overkill? It doesn’t take a lot of work to add proto buffers to an existing project (if you’re already using a reasonable build system, anyway).
Re: A new Go API for Protocol Buffers
#37Not to be too Rust evangelist here, but why the emphasis on reflection instead of further support for generics? On the Rust side, macros can[1] transform .proto files into data structure definitions at compile time, and instances of the structs passed around have the benefit of strong typing, code completion in editors, lints and tests can easily validate properties of protobuf objects and so on. And all without addi…
Whether a field is "sensitive" may change after the field is defined and used all over the place, so you can't just move it without changing all the code. But you can easily add a option (like [non_sensitive=true]) to any field so your redaction code picks it up.
Re: A new Go API for Protocol Buffers
#38> The github.com/golang/protobuf module is APIv1. > The google.golang.org/protobuf module is APIv2. We have taken advantage of the need to change the import path to switch to one that is not tied to a specific hosting provider. That's such a weird choice, and will be quite confusing. Now every time you see protobuf being imported, you have to be sure to pay extra attention to the domain used, and correctly remember w…
The goal of issuing a major version is to create a separate package namespace so it can be imported concurrently to other major versions. By treating major versions as different packages, it creates efficiencies in VCS history and common namespace. However, the goal is to create a new package name that can be concurrently imported with previous versions. Everything else is just method. Don't confuse the goal with the…
Re: A new Go API for Protocol Buffers
#39Earlier quoted context omitted.
What are you going to do for APIv3? Version it as google.golang.org@v2? google.golang.org@v3 (but @v2 doesn't exist)? Release an @v2 which is identical to @v1?
We used up our breaking change budget for the next decade with this release, so we'll think about it in 2030.
Re: A new Go API for Protocol Buffers
#40> The github.com/golang/protobuf module is APIv1. > The google.golang.org/protobuf module is APIv2. We have taken advantage of the need to change the import path to switch to one that is not tied to a specific hosting provider. That's such a weird choice, and will be quite confusing. Now every time you see protobuf being imported, you have to be sure to pay extra attention to the domain used, and correctly remember w…
> If only go modules established a clear way to differentiate API versions using the import path! It did, they just chose not to follow it :/ I wonder what rsc thinks about this decisions