21–30 of 245 posts
I wonder if we can write an API w/ JSON the usual way and change the final packaging to send it over protobuf.
I was pushing at one point for us to have some code in our protobuf parsers that would essentially allow reading messages in either JSON or binary format, though to be fair there's some overhead that way by doing some kind of try/catch, but, for some use cases I think it's worth it...
One of the best parts of Protobuf is that there's a fully compatible JSON serialization and deserialization spec, so you can offer a parallel JSON API with minimal extra work.
Idk I built a production system and ensured all data transfers, client to server and server to client were proto buf and it was a pain. Technically, it sounds really good but the actual act of managing it is hell. That or I need a lot of practice to use them, at that point shouldn't I just use JSON and get on with my life.
Making changes to messages in a backwards-compatible way can be annoying, but JSON allowing you to shoot yourself in the foot will take more time and effort to fix when it's corrupting data in prod than protobuf giving you a compile error would.
Sure it will blow up in your face when a field goes missing or value changes type.
People who advocate paying the higher cost ahead of time to perfectly type the entire data structure AND propose a process to do perform version updates to sync client/server are going to lose most of the time.
The zero cost of starting with JSON is too compelling even if it has a higher total cost due to production bugs later on.
When judging which alternative will succeed, lower perceived human cost beats lower machine cost every time.
This is why JSON is never going away, until it gets replaced with something with even lower human communication cost.
Is it just me or is this article insanely confusing? With all due respect to the author, please be mindful of copy editing LLM-assisted writing. There is a really interesting discussion underneath of this as to the limitations of JSON along with potential alternatives, but I can't help but distrust this writing due to how much it sounds like an LLM.
I didn't find it confusing.
I found it unconvincing, but the argument itself was pretty clear. I just disagreed with it.
Mandatory comment about ASN.1, a protocol from 1984, already did what Protobuf does, with more flexibility. Yes, it's a bit ugly but if you stick to the DER encoding it's really not worse than Protbuf at all. Check out the Wikipedia example: https://en.wikipedia.org/wiki/ASN.1#Example_encoded_in_DER Protobuf is ok but if you actually look at how the serializers work, it's just too complex for what it achieves.
After working heavily with SNMP across a wide variety of OEMs, this flexibility becomes a downside. Or SNMP/MIBs were specified at the wrong abstraction level, where the ASN.1 flexibility gives mfgs too much power to do insane and unconventional things.
"Better than JSON" is a pretty bold claim, and even though the article makes some great cases, the author is making some trade-offs that I wouldn't make, based on my 20+ year career and experience. The author makes a statement at the beginning: "I find it surprising that JSON is so omnipresent when there are far more efficient alternatives." We might disagree on what "efficient" means. OP is focusing on computer effi…
In terms of human effort, a strongly typed schema rather than one where you have to sanity check everything saves far more time in the long run.
Mandatory comment about ASN.1, a protocol from 1984, already did what Protobuf does, with more flexibility. Yes, it's a bit ugly but if you stick to the DER encoding it's really not worse than Protbuf at all. Check out the Wikipedia example: https://en.wikipedia.org/wiki/ASN.1#Example_encoded_in_DER Protobuf is ok but if you actually look at how the serializers work, it's just too complex for what it achieves.
It's premature and maybe presumptuous of him to be advertising protobufs when he hasn't heard of the alternatives yet. I'll engage the article after he discovers them...
> With Protobuf, that’s impossible. Unless your servers and clients push at different time, thus are compiled with different versions of your specs, then many safety bets are off. There are ways to be mostly safe (never reuse IDs, use unknown-field-friendly copying methods, etc.), but distributed systems are distributed systems, and protobuf isn't a silver bullet that can solve all problems on author's list. On the u…