Live data from Hacker News

Protocol Buffers v3.0.0 released

github.com

21–30 of 131 posts

Re: Protocol Buffers v3.0.0 released

#21

If someone better informed than me can please explain - where and why would something like Protocol Buffers be useful?

We use them internally at Square for our RPC mechanism ("Sake", similar to "Stubby", Google's internal RPC mechanism), for our Kafka-based logging/metrics/queue infrastructure, and for defining external JSON APIs. We're in the process of switching from Sake to GRPC, which also use Protobufs as their payload format (although you can sub in different transports).

Re: Protocol Buffers v3.0.0 released

#22

If someone better informed than me can please explain - where and why would something like Protocol Buffers be useful?

I used it in a trading system because it's a compact scheme for sending data across networks. It's also quite fast, and there's support for various languages. So you can have a feed handler blasting out prices using a c++ implementation, with a GUI drawing a chart written in c#.

Re: Protocol Buffers v3.0.0 released

#23

Google also has flatbuffers. I wonder if flatbuffers is being used by enough developers to justify significant development? https://github.com/google/flatbuffers

Been using flatbuffers in production for a high speed market feed for a month now. Love it. Decode/encode time is absurdly fast (~1-2 microseconds for a small to medium schema). If you're pushing 50k+ events/second it can be a great choice. Takes up almost no space on the wire too.

Re: Protocol Buffers v3.0.0 released

#24
post #15

Earlier quoted context omitted.

> Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. From https://developers.google…

Yes, I read this. It tells me what Protocol Buffers are. Faster, Smaller XML like data structures for serialisation. What are the most common use cases though? And do people only use them for performance reasons?

As an example, they power Google's RPC system (usually referred to as Stubby in the literature, recently open sourced as gRPC http://www.grpc.io/)

Re: Protocol Buffers v3.0.0 released

#26
post #5

"The main intent of introducing proto3 is to clean up protobuf before pushing the language as the foundation of Google's new API platform" Does anyone know if this means Google's public APIs will be proto3 based? I quite like protobufs.

They've been experimenting[1] with exposing Google Cloud Platform APIs over gRPC (which is powered by proto3), so it seems quite likely.

[1] https://cloud.google.com/blog/big-data/2016/03/announcing-gr...

Re: Protocol Buffers v3.0.0 released

#27

If someone better informed than me can please explain - where and why would something like Protocol Buffers be useful?

Serializing data for RPC, network protocols or storage, description and serialization of configuration, serializable state, serializing complex types for cryptographic signing, etc.

Why is it useful? The schema both documents the data structure and allows mappings to natural APIs in many different languages. Parsers and encoders are generated for you, and are fast.

Re: Protocol Buffers v3.0.0 released

#28

Google also has flatbuffers. I wonder if flatbuffers is being used by enough developers to justify significant development? https://github.com/google/flatbuffers

I think it's more that GRPC (Google's RPC-over-HTTP2 protocol) directly supports Protobuf, and not Flatbuffers. All of Google's Cloud APIs use Protobuf (for example the [Speech API]( https://cloud.google.com/speech/reference/rpc/ ) ). I have to say, GRPC is pretty great. It's statically typed, supports loads of languages, the interfaces are simple to define (basically Protobuf), and it supports streaming requests! Mo…

As an FYI, GRPC support was added to flatbuffers a month ago. https://github.com/google/flatbuffers/tree/master/grpc

Re: Protocol Buffers v3.0.0 released

#29

> primitive fields set to default values (0 for numeric fields, empty for string/bytes fields) will be skipped during serialization. I don't totally understand this. Presumably during deserialization they will be set to defaults and not missing? Otherwise, coupled with the removal of required fields, it seems impossible to actually send a 0-value number or empty string, or to send a proto without a field and not have…

Within the API, proto3 does not have the concept of field presence. All fields are "present" and default to their type's zero value. Since the client can handle this, there is no need to explicitly serialize default values.

and how do you send a explicit zero so that the client knows that the field is really set by the server and not the default? or a explicit empty string?
Post reply on HN