If someone better informed than me can please explain - where and why would something like Protocol Buffers be useful?
When JSON is not fast/lightweight enough.
Protocol Buffers v3.0.0 released
51–60 of 131 posts
Re: Protocol Buffers v3.0.0 released
#52- removing optional values is actually quite nice. In practice, I end up checking for "missing or empty string" anyway. - the "well-known types" boxed primitive types essentially add optional values back in. And depending on your language bindings, may look the same. - extensions are still allowed in proto3 syntax files, but only for options - since the descriptor is still proto2. It seems odd to build a proto3 that…
> removing optional values Slight correction: optional values are not removed. Quite the opposite; the "optional" keyword is removed because now all fields are optional. It is actually required values which were removed.
You're both right. What has been removed is the concept of presences altogether.
Re: Protocol Buffers v3.0.0 released
#53Earlier quoted context omitted.
As an example, they power Google's RPC system (usually referred to as Stubby in the literature, recently open sourced as gRPC http://www.grpc.io/ )
gRPC is based on Stubby: http://www.grpc.io/posts/principles
Stubby is based on protobuf; where by "based" I mean layered on, i.e. protobuf is the encoding used by stubby to encode requests and responses.
gRPC is a reimplementation of Stubby suitable to be used outside of Google.
Re: Protocol Buffers v3.0.0 released
#54Earlier quoted context omitted.
and how do you send a explicit zero so that the client knows that the field is really set by the server and not the default? or a explicit empty string?
One case where this question is important is when you are updating a record stored by the server. You only want to send fields you are changing because the record might be huge. But then how does the server distinguish between fields you didn't set and fields you want to set back to the default? The solution is to also tell the server which fields you are changing in a separate message. Example: { 'update_record': {…
Thanks for explaining that and for the reference. It seems like a lot of overhead for a protocol that is designed to be cheap...
Re: Protocol Buffers v3.0.0 released
#55Earlier quoted context omitted.
Been using flatbuffers in production for a high speed market feed for a month now. Love it. Decode/encode time is absurdly fast (~1-2 microseconds for a small to medium schema). If you're pushing 50k+ events/second it can be a great choice. Takes up almost no space on the wire too.
Try Cap'n Proto instead. Better designed and faster.
Kenton Varda (the author of Cap'n Proto), was the primary author of Protocol Buffers version 2.
Quoting from https://capnproto.org/index.html
Because it’s easy to pick on myself. :) I, Kenton Varda, was
the primary author of Protocol Buffers version 2, which is the
version that Google released open source. Cap’n Proto is the
result of years of experience working on Protobufs, listening
to user feedback, and thinking about how things could be done
better.Re: Protocol Buffers v3.0.0 released
#56They added a feature that impressively fails to interoperate with the rest of the world. > Added well-known type protos (any.proto, empty.proto, timestamp.proto, duration.proto, etc.). Users can import and use these protos just like regular proto files. Additional runtime support are available for each language. From timestamp.proto: // A Timestamp represents a point in time independent of any time zone // or calenda…
It's interesting that you refer to a huge amount of planning and engineering as "sticking your head in the sand".
https://googleblog.blogspot.com/2011/09/time-technology-and-...
I think that the approach everything else uses is the "sticking your head in the sand approach". You basically pretend that there is no problem and that time is perfectly accurate, up until you have a minute with 59 or 61 seconds.
Just because suddenly trying to handle "Oh shit, everything is off by an entire second!" is the approach everything else uses doesn't mean it is the right approach.
Re: Protocol Buffers v3.0.0 released
#57They added a feature that impressively fails to interoperate with the rest of the world. > Added well-known type protos (any.proto, empty.proto, timestamp.proto, duration.proto, etc.). Users can import and use these protos just like regular proto files. Additional runtime support are available for each language. From timestamp.proto: // A Timestamp represents a point in time independent of any time zone // or calenda…
I can see caring about leap seconds right now, but a few seconds back or forth in the past probably won't matter very much.
Re: Protocol Buffers v3.0.0 released
#58- removing optional values is actually quite nice. In practice, I end up checking for "missing or empty string" anyway. - the "well-known types" boxed primitive types essentially add optional values back in. And depending on your language bindings, may look the same. - extensions are still allowed in proto3 syntax files, but only for options - since the descriptor is still proto2. It seems odd to build a proto3 that…
> - removing optional values is actually quite nice. In practice, I end up checking for "missing or empty string" anyway.
I feel the opposite; this greatly reduces the utility of protobuf.
Previously, I could trust that if parsing succeeded, then I had a guarantee of a populated data structure.
Now, I have to check each field individually, in manually written code, to verify that no required fields are missing.
That's really lame, and a huge step backwards.
Re: Protocol Buffers v3.0.0 released
#59Earlier quoted context omitted.
> Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. From https://developers.google…
Yes, I read this. It tells me what Protocol Buffers are. Faster, Smaller XML like data structures for serialisation. What are the most common use cases though? And do people only use them for performance reasons?
In addition to smaller/faster than XML, protobufs make it extremely easy to declare the schema of data, validate data and version your schema. Then the generated wrappers and static type checking in various languages add additional guarantees that you're using the data correctly.
Plain XML still requires a lot to ensure compatibility when it's used across multiple places, protobufs attempt to minimize many sources of the incompatibilities.
Add in a bunch of tools such as protobuf->JSON, protobuf plaintext serialization, etc and it becomes more difficult to argue for using something such as XML or vanilla JSON.
Flatbuffers are still a nice solution for more performance-critical applications.
Re: Protocol Buffers v3.0.0 released
#60They added a feature that impressively fails to interoperate with the rest of the world. > Added well-known type protos (any.proto, empty.proto, timestamp.proto, duration.proto, etc.). Users can import and use these protos just like regular proto files. Additional runtime support are available for each language. From timestamp.proto: // A Timestamp represents a point in time independent of any time zone // or calenda…
[deleted]