Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Protobuffers Are Wrong (2018)

reasonablypolymorphic.com

Post reply on HN
Print view View original

81–90 of 321 posts

Re: Protobuffers Are Wrong (2018)

#81

Post by zigzag312 »

jitl wrote: post #19
lalaithion wrote: post #7

Protocol buffers suck but so does everything else. Name another serialization declaration format that both (a) defines which changes can be make backwards-compatibly, and (b) has a linter that enforces backwards compatible changes. Just with those two criteria you’re down to, like, six formats at most, of which Protocol Buffers is the most widely used. And I know the article says no one uses the backwards compatible…

Not widely used but I like Typical's approach https://github.com/stepchowfun/typical > Typical offers a new solution ("asymmetric" fields) to the classic problem of how to safely add or remove fields in record types without breaking compatibility. The concept of asymmetric fields also solves the dual problem of how to preserve compatibility when adding or removing cases in sum types.

This actually looks quite interesting.

Re: Protobuffers Are Wrong (2018)

#82

Post by thinkharderdev »

iamdelirium wrote: post #58
thinkharderdev wrote: post #48

Earlier quoted context omitted.

Yeah, as soon as you have a moderately complex type the generated code is basically useless. Honestly, ~80% of my gripes about protocol buffers could be alleviated by just allowing me to mark a message field as required.

You think you do but you really don't. What happens if you mark a field as required and then you need to delete it in the future? You can't because if someone stored that proto somewhere and is no longer seeing the field, you just broke their code.

If you need to deserialize an old version then it's not a problem. The unknown field is just ignored during deserialization. The problem is adding a required field since some clients might be sending the old value during the rollout.

But in some situations you can be pretty confident that a field will be required always. And if you turn out to be wrong then it's not a huge deal. You add the new field as optional first (with all upgraded clients setting the value) and then once that is rolled out you make it required.

And if a field is in fact semantically required (like the API cannot process a request without the data in a field) then making it optional at the interface level doesn't really solve anything. The message will get deserialized but if the field is not set it's just an immediate error which doesn't seem much worse to me than a deserialization error.

Re: Protobuffers Are Wrong (2018)

#83

Post by taeric »

ozgrakkurt wrote: post #75
iamdelirium wrote: post #58

Earlier quoted context omitted.

You think you do but you really don't. What happens if you mark a field as required and then you need to delete it in the future? You can't because if someone stored that proto somewhere and is no longer seeing the field, you just broke their code.

Maybe you don’t delete it then?

I mean, this is essentially the same lesson that database admins learn with nullable fields. Often it isn't the "deleting one is hard" so much as "adding one can be costly."

It isn't that you can't do it. But the code side of the equation is the cheap side.

Re: Protobuffers Are Wrong (2018)

#84

Post by seanw444 »

xmddmx wrote: post #15

I share the author's sentiment. I hate these things. True story: trying to reverse engineer macOS Photos.app sqlite database format to extract human-readable location data from an image. I eventually figured it out, but it was: A base64 encoded Binary Plist format with one field containing a ProtoBuffer which contained another protobuffer which contained a unicode string which contained improperly encoded data (for e…

That's horrendous. For some reason I imagine Apple's software to be much cleaner, but I guess that's just the marketing getting to my head. Under the hood it's still the same spaghetti.

Re: Protobuffers Are Wrong (2018)

#85

Post by perching_aix »

jilles wrote: post #8
ndr wrote: post #3

Not even before the first line ends you get "They’re clearly written by amateurs". This is a rage bait, not worth the read.

The best way to get your point across is by starting with ad-hominem attacks to assert your superior intelligence.

Is this in reference to the blogpost, the comment above, or your own comment? Cause it honestly works for all of them.

Re: Protobuffers Are Wrong (2018)

#87

Post by guzik »

We thought for a long time about using protobufs in our product [1] and in the end we went with JSON-RPC 2.0 over BLE, base64 for bigger chunks. Yeah, you still need to pass sample format and decode manually. The overhead is fine tho, debugging is way easier (also pulling in all of protobuf just wasn't fun).

[1] aidlab.com/aidlab-2

Re: Protobuffers Are Wrong (2018)

#88

Post by swiftcoder »

tyleo wrote: post #57
lalaithion wrote: post #7

Protocol buffers suck but so does everything else. Name another serialization declaration format that both (a) defines which changes can be make backwards-compatibly, and (b) has a linter that enforces backwards compatible changes. Just with those two criteria you’re down to, like, six formats at most, of which Protocol Buffers is the most widely used. And I know the article says no one uses the backwards compatible…

We use protocol buffers on a game and we use the back compat stuff all the time. We include a version number with each release of the game. If we change a proto we add new fields and deprecate old ones and increment the version. We use the version number to run a series of steps on each proto to upgrade old fields to new ones.

> We use the version number to run a series of steps on each proto to upgrade old fields to new ones

It sounds like you've built your own back-compat functionality on top of protobuf?

The only functionality protobuf is giving you here is optional-by-default (and mandatory version numbers, but most wire formats require that)

Re: Protobuffers Are Wrong (2018)

#90

Post by bloppe »

Protobuf's main design goal is to make space-optimized binary tag-length-value encoding easy. The mentality is kinda like "who cares what the API looks like as long as it can support anything you want to do with TLV encoding and has great performance." Things like oneofs and maps are best understood as slightly different ways of creating TLV fields in a message, rather than pieces of a comprehensive modern type system. The provided types are simply the necessary and sufficient elements to model any fuller type system using TLV.
Post reply on HN