Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Arguing against using protobuffers

reasonablypolymorphic.com

Post reply on HN
Print view View original

201–210 of 307 posts

Re: Arguing against using protobuffers

#201

Post by hardwaresofton »

hardwaresofton wrote: post #67
atombender wrote: post #25

Though I dislike the hyperbolic tone and personal attacks, the author isn't entirely wrong. There are many design choices in Protocol Buffers that seem directly related to the scale and complexity at which Google operates, and which sacrifice safety, clarity and language integration. The utter awkwardness of Protobuf-generated code is particularly problematic. I've had pretty good results with the TypeScript code gen…

I'd like to humbly suggest that we use JSON please, in particular: JSON + JSONSchema[0] +/- JSON Hyperschema[1] +/- JSON LD[2] It's a bit to learn but I promise you, it's worth it. The technologies are not redundant (jsonschema spec is for validation, hyperschema spec is for specifying how you interact, and LD is for semantics like language and more). If you take a few hours, read all 3 specs, you're almost guarantee…

Small correction -- I meant vertical filtering in postgrest[0]

[0]: https://postgrest.org/en/v5.1/api.html#vertical-filtering-co...

Re: Arguing against using protobuffers

#202

Post by steve-chavez »

hardwaresofton wrote: post #190
camgunz wrote: post #188

Earlier quoted context omitted.

GraphQL is more or less a data description language. It describes queries and mutations. There are JS libraries that take a GraphQL query/mutation, convert it into the spec'd JSON payload, send it to the backend (single endpoint), and map the requested fields back into a JS object to be used. This is super nice when paired with React -- it's hard to overstate how nice it is. On the backend, there are libraries that p…

This is an excellent response and is 100% correct. At this point all I can say about my stance is that I would have preferred a standard for HTTP+JSON instead of how GraphQL is completely different. I also completely forgot about HAL[0]. To be clear, I'm not against GraphQL, I understand it and I understand what it brings to the table but I just thought the alternative was better if only because it relied on a web of…

I've long thought about creating a standard(iana type) for PostgREST json schema + http querystring conventions. I've seen some APIs[1](only one I can remember now) that follow PostgREST conventions, so perhaps this could benefit all of us who don't want to jump on the GraphQL bandwagon but still want an expressive and interoperable/standardized way to query resources.

We currently use OpenAPI but we found some shortcomings[2] and would like to have a more tailored standard.

Right now what we're missing is more funding, we don't have a backing company like GraphQL/gRPC, we rely on community donations through Patreon[3]. Definitely consider supporting us if you want to improve the state of REST APIs, allow us to keep fighting for web standards :).

[1]:https://developers.blockapps.net/advanced/concepts/#cirrus

[2]:https://github.com/PostgREST/postgrest/issues/790#issuecomme...

[3]:https://www.patreon.com/postgrest

Re: Arguing against using protobuffers

#203

Post by stouset »

asdfasgasdgasdg wrote: post #187
stouset wrote: post #183

Earlier quoted context omitted.

Absolutely no to JSON, IMO. The problem with JSON for internal formats is that there’s often only one consumer and producer, so documenting the format rarely happens. Later, when you want to reimplement one side, you learn that there is no “one place” where you parse the JSON, but that you hand bits and pieces of it to completely unrelated areas of code. Figuring out these as hoc formats is nigh impossible, so you en…

This is the classic static vs dynamic typing question. Suffice to say, I don't think there is sufficient science or anecdata on the issue to really give a solid answer.

I am a proponent of both static (Rust) and dynamic (Ruby) typing in programming languages. They both have their place, and if there are bugs you can always fix the code.

The one place where I don’t think dynamic types have their place is in internal interchange formats. These can live forever, and they can change subtly over time. It’s the living forever part that changes the calculus.

Re: Arguing against using protobuffers

#204

Post by deorder »

atombender wrote: post #116
hardwaresofton wrote: post #103

Earlier quoted context omitted.

Nothing about what you just posted couldn't be done with a normal RESTful endpoint with sufficient support for column-level filtering and embedded item filtering. Your post is the perfect example of GraphQL is being over hyped. I absolutely get that there's a benefit to filtering at both these levels, and that the DSL cuts down on noise and gives you a way to "query" without thinking of web requests, but it's not a l…

Could one invent an ad-hoc REST API to perform joins, apply parameters and so forth? Of course. That's what people have been doing. That means every solution is ad-hoc. People get tired of this. Have you tried any of the GraphQL client tools? You can literally point a client at an arbitrary server and not only see its schema, but also interact with the API. REST doesn't give you that, because the authors of REST negl…

You can do this with PostgReST as well. Something like:

  GET /user?select=name,photo(url),friend(name)
I do not use PostgReST, but created something similar.

When I perform the following 3 individual requests after I performed the above request the results can all be retrieved from cache (at the server and/or client side):

  GET /user?select=name
  GET /user/{name}/photo?select=url
  GET /user/{name}/friend?select=name
Even when I do the following (without the photo(url) part) it can now just build it from the cache without hitting the database server:

GET /user?select=name,friend(name)

That is one of the advantages of using ReST. It allows you to utilize and benefit from already existing proven and well defined infrastructure components (the connectors etc.).

GraphQL can be used on top of this as well. I experimented with this once as an alternative to the PostgReST-like syntax.

Re: Arguing against using protobuffers

#205

Post by AstralStorm »

malkia wrote: post #175
derefr wrote: post #111

Earlier quoted context omitted.

The author isn’t trying to say either of those things, really. I think their real point was something like: many companies that have an Enterprise Service Bus architecture for their pile of polyglot microservices, have a dogma for the encoding of the data flowing over the Bus. And Protobufs, though good for some use-cases, are a particularly bad dogma to be stuck with. That is, when they don’t work for a use-case, th…

What makes you say that Protobufs are bad thing? I'm finding the opposite - where you end up with everyone's favourite json parser, not able to handle UTF8 here and there, or some handle multi-line json, other handle comments, etc, etc. Also certainly slower, and certainly when you process data back you have to serialize, convert, check, etc. For me protobuf+grpc gives you enough building block, not to shoot yourself…

It doesn't complicate just build systems, because of the weaknesses of it's type system and code generation it requires wrappers. Those get desynced and take maintenance.

This is the main point where the whole reason to use Protobuf falls apart - it doesn't work well enough to skip manual parsing.

And if you do manual parsing you might well go all the way in and use JSON with a schema and a reasonable library to handle these. You gain very little convenience wise and only a bunch of line bytes at best. Which typically does not matter for use cases of Protobuf that are not Google sized.

Re: Arguing against using protobuffers

#207

Post by reacweb »

malkia wrote: post #61

I spent 2.5 years at Google, and most of what I did was pushing one protobuf from one place to another :) - and I loved it... Honestly though, you can complain all day, but some of the decisions made in the list you presented most likely come from experience (daily) not as a user of protobufs (which I simply was), but someone that had to support a plethora of compression formats, how protobufs gets stored in the diff…

It is the first time I work on a project using protobuf. Previously, I have used corba, asn1 and xml. Protobuf (in java) is a real pleasure to work with.

Re: Arguing against using protobuffers

#208

Post by _Codemonkeyism »

oh_sigh wrote: post #129
dweis wrote: post #60

Hi there, I'm an actual author of Protocol Buffers :) I think Sandy's analysis would benefit from considering why Protocol Buffers behave the way they do rather than outright attacking the design because it doesn't appear to make sense from a PL-centric perspective. As with all software systems, there are a number of competing constraints that have been weighed that have led to compromises. - D P.S. I also don't beli…

I thought Kenton Varda was the author of protobufs?

"an actual author"

Re: Arguing against using protobuffers

#209

Post by vkou »

docker_up wrote: post #193
taeric wrote: post #161

Earlier quoted context omitted.

You are missing when you have a middle layer. Message comes in at v3 and hits a layer that only knows v1 then gets passed to a layer that is at v4. I'd wager most places don't have that many layers. But, if you are embracing microservices, you'll find yourself here fairly fast.

I doubt any api call trying to work in such a chaotic environment would actually work, and having all optional fields won't magically make things work. It will probably fail but in very mysterious ways. This sounds more like an environment where microservices are completely out of control and chaotic.

Doubt all you will, but being able to update different pieces of your service independently is a huge win. Having to update everything in lockstep is what I'd describe as 'out of control'. Which is precisely what forcing every field to be required does.

There's a million and one reasons for why you may want to push, or rollback only 1 out of X services. If you follow a few simple rules when adding/removing fields, you can do this safely.

Re: Arguing against using protobuffers

#210

Post by UncleMeat »

twinkletwinkle wrote: post #85
the_other_guy wrote: post #55

Yet protobuf is probably the most compact, efficient and performant serialization method especially when saving bandwidth is important. I experimented with protofbuf, flatbuffers and messagepack and always found protobuf messages the most compact by a noticeable margin

That's the core of the author's argument. Protobuffers optimize for something besides usability and maintainability, because Google cares more about incremental performance than developer-friendliness. Which is a fine thing to care about at Google's scale, but maybe others' calculations should be different.

That's a fine argument. But then the author decides to call the people who wrote protobuf idiots and amateurs.
Post reply on HN