The reason to use Protobuf is not performance
1–10 of 23 posts
Re: The reason to use Protobuf is not performance
#2schema-driven development is great, but I wish people spent more time on JSON-based API schema rather than switch to protobuf. Because protobufs immediately require some minimal level of complexity (at least a build system!) and thus effectively block small one-off scripts. And the debuggability is also not great.
Can we (as an industry) pick a standard (openapi? jsonschema? something else?) and add the tooling to match protobuf ecosystem functionality?
Re: The reason to use Protobuf is not performance
#3schema-driven development is great, but I wish people spent more time on JSON-based API schema rather than switch to protobuf. Because protobufs immediately require some minimal level of complexity (at least a build system!) and thus effectively block small one-off scripts. And the debuggability is also not great. Can we (as an industry) pick a standard (openapi? jsonschema? something else?) and add the tooling to ma…
I don't think a large group can pick a standard. What works well for one programming language does not work for another, and we devolve into tribes. Protobuf gained popularity when it dropped features to better support Go more naturally (no more null/nil for example), and rode in on a wave of Go adoption. And made things worse from a Python developer perspective. Pretty much the same as toml adoption, which fits more naturally with Go and more static languages, unlike the much more dynamic and flexible yaml.
Re: The reason to use Protobuf is not performance
#4schema-driven development is great, but I wish people spent more time on JSON-based API schema rather than switch to protobuf. Because protobufs immediately require some minimal level of complexity (at least a build system!) and thus effectively block small one-off scripts. And the debuggability is also not great. Can we (as an industry) pick a standard (openapi? jsonschema? something else?) and add the tooling to ma…
GRPC Rest transcoding works fine, you can do this with envoy for example.
And if you use the connect protocol (also from buf) then you can also use json.
So it's quite achievable to get all the benefits you want from the gRPC ecosystem for developing your service while also allowing quick and dirty client usage (i.e. scripting with cURL).
Re: The reason to use Protobuf is not performance
#5Inbound marketing is so hard to take seriously.
Re: The reason to use Protobuf is not performance
#6schema-driven development is great, but I wish people spent more time on JSON-based API schema rather than switch to protobuf. Because protobufs immediately require some minimal level of complexity (at least a build system!) and thus effectively block small one-off scripts. And the debuggability is also not great. Can we (as an industry) pick a standard (openapi? jsonschema? something else?) and add the tooling to ma…
I don't think a large group can pick a standard. What works well for one programming language does not work for another, and we devolve into tribes. Protobuf gained popularity when it dropped features to better support Go more naturally (no more null/nil for example), and rode in on a wave of Go adoption. And made things worse from a Python developer perspective. Pretty much the same as toml adoption, which fits more…
> I don't think a large group can pick a standard.
There are plenty of cases where this has happened. JSON has already become the defacto standard. It's not a the step to formalize a ubiquitous format has been taken many times. Every major platform supports it or uses it exclusively. Some frameworks and tooling, only uses JSON (true of other formats to an extent, ie json more than yaml).
> What works well for one programming language does not work for another,
I'm not sure why this roundabout whataboutism is compelling. A standard protocol is used in absence of other restrictions or benefits of another This doesn't snap away the other protocols from existence.
Re: The reason to use Protobuf is not performance
#7While certainly not as ubiquitous as Protobuf, Cap'n Proto[1] is an extremely excellent alternative worth checking out. The quality of the generated C++ code is enough reason for me to prefer it.
Re: The reason to use Protobuf is not performance
#8schema-driven development is great, but I wish people spent more time on JSON-based API schema rather than switch to protobuf. Because protobufs immediately require some minimal level of complexity (at least a build system!) and thus effectively block small one-off scripts. And the debuggability is also not great. Can we (as an industry) pick a standard (openapi? jsonschema? something else?) and add the tooling to ma…
I fail to see how running something like protoc before running or compiling code is a show stopper. There are even npm packages that will install protoc for you [1]. That means you can use it in a custom npm build script action, or invoke it before staring the dev server? Put in a git pre-commit to run it to assert there are no changes? Run via a file-watcher?
IMO writing a schema in something like JSON for OpenAPI is way more verbose and time consuming than writing a proto file and a script or Makefile to invoke protoc. There also is the mismatch in capabilities. The JS-adjacent schema standards mostly do not directly support native types like uint64, let alone directly specify how to encode numbers like fixed64. I mean, look... the work around in JavaScript still is use "string" [2].
On the other hand, there is a standard in OpenAPI for validating values. But that doesn't make sense in the Protobuf world since usually the API is expected to be forward and backward compatible, within reason, so something like a min and max value needs to be done at runtime, and not fixed within the schema. And don't get me started on trying to define cross-field validation in a declarative schema. It is easier to just write the code. :-)
So, IMO, no, a single standard won't happen in the industry. The many dimensions like logical vs physical models, validation, or error handling that make it impossible.
[1] https://www.npmjs.com/package/@protobuf-ts/protoc [2] https://github.com/OAI/OpenAPI-Specification/issues/2617#iss...
Re: The reason to use Protobuf is not performance
#9While certainly not as ubiquitous as Protobuf, Cap'n Proto[1] is an extremely excellent alternative worth checking out. The quality of the generated C++ code is enough reason for me to prefer it. [1] https://capnproto.org/
Does protobuf or capnproto have good tooling support out in the world?
Proto was very convenient at Google since everything was proto, your debug tools were proto, all your data pipeline tools spoke proto, you had command line utilities for protos, etc, etc.
It seems like the rest of the world is on JSON unless they actually need performance, so if you don't get performance, should you use something like a code generator for JSON types like Conjure so that you don't actually have to deal with these binary formats everywhere?
Re: The reason to use Protobuf is not performance
#10schema-driven development is great, but I wish people spent more time on JSON-based API schema rather than switch to protobuf. Because protobufs immediately require some minimal level of complexity (at least a build system!) and thus effectively block small one-off scripts. And the debuggability is also not great. Can we (as an industry) pick a standard (openapi? jsonschema? something else?) and add the tooling to ma…
I fail to see how running something like protoc before running or compiling code is a show stopper. There are even npm packages that will install protoc for you [1]. That means you can use it in a custom npm build script action, or invoke it before staring the dev server? Put in a git pre-commit to run it to assert there are no changes? Run via a file-watcher? IMO writing a schema in something like JSON for OpenAPI i…
[deleted]