Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I stopped using JSON for my APIs

aloisdeniel.com

Post reply on HN
Print view View original

81–90 of 245 posts

Re: Why I stopped using JSON for my APIs

#81

Post by theamk »

brabel wrote: post #15

Mandatory comment about ASN.1, a protocol from 1984, already did what Protobuf does, with more flexibility. Yes, it's a bit ugly but if you stick to the DER encoding it's really not worse than Protbuf at all. Check out the Wikipedia example: https://en.wikipedia.org/wiki/ASN.1#Example_encoded_in_DER Protobuf is ok but if you actually look at how the serializers work, it's just too complex for what it achieves.

ASN.1 has too much stuff. The moment you write "I made ASN.1 decoder/encoder", someone will throw TeletexString or BMPString at it. Or inheritance, as morshu9001 sad. So at this point:

- You can support all those features, and your ASM.1 library will be horribly bloated and over-engineered.

- You can support your favorite subset, but then you cannot say it's ASN.1 anymore. It will be "ASN.brabel", which only has one implementation (yours). And who wants that?

(unless you are Google and have immense developer influence... But in this case, why not design things from scratch, since we are making all-new protocol anyway?)

Re: Why I stopped using JSON for my APIs

#82

Post by ses1984 »

strongpigeon wrote: post #36
brabel wrote: post #15

Mandatory comment about ASN.1, a protocol from 1984, already did what Protobuf does, with more flexibility. Yes, it's a bit ugly but if you stick to the DER encoding it's really not worse than Protbuf at all. Check out the Wikipedia example: https://en.wikipedia.org/wiki/ASN.1#Example_encoded_in_DER Protobuf is ok but if you actually look at how the serializers work, it's just too complex for what it achieves.

> Protobuf is ok but if you actually look at how the serializers work, it's just too complex for what it achieves. Yeah. I do remember a lot of workloads at Google where most of the CPU time was spent serializing/deserializing protos.

I feel like most high throughput distributed systems eventually reach a point where some part of it is constrained by de/serialization.

Not much is faster than protobuf except for zero copy formats.

Re: Why I stopped using JSON for my APIs

#83

Post by shortrounddev2 »

klysm wrote: post #70
dfabulich wrote: post #68

> With JSON, you often send ambiguous or non-guaranteed data. You may encounter a missing field, an incorrect type, a typo in a key, or simply an undocumented structure. With Protobuf, that’s impossible. Everything starts with a .proto file that defines the structure of messages precisely. This deeply misunderstands the philosophy of Protobuf. proto3 doesn't even support required fields. https://protobuf.dev/best-pra…

It’s also conflating the serialization format with contracts

Most web frameworks do both at the same time to the point where having to write code which enforced a type contract after deserializing is a delabreaker for me. I eant to be able to define my DTOs in one place, once, and have it both deserialize and enforce types/format. Anything else is code smell

Re: Why I stopped using JSON for my APIs

#84

Post by rileymichael »

c-cube wrote: post #55
morshu9001 wrote: post #41

Protos are great. Last time I did a small project in NodeJS, I set up a server that defines the entire API in a .proto and serves each endpoint as either proto or json, depending on the content header. Even if the clients want to use json, at least I can define the whole API in proto spec instead of something like Swagger. So my question is, why didn't Google just provide that as a library? The setup wasn't hard but…

That's what Twirp ( https://github.com/twitchtv/twirp ) is about. Protobuf or JSON, over any HTTP, with a simple URL schema. It's fairly simple.

highly recommend twirp, even in current year. connectrpc seems to have stalled so there isn't a ton of languages w/server support, and because of the grpc interop on top of their own protocol it's a bit of an undertaking to roll your own.

the twirp spec however is so simple you can throw together your own code generator in an afternoon for whatever language you want.

Re: Why I stopped using JSON for my APIs

#85

Post by duped »

barremian wrote: post #56
pyrolistical wrote: post #24

Compressed JSON is good enough and requires less human communication initially. Sure it will blow up in your face when a field goes missing or value changes type. People who advocate paying the higher cost ahead of time to perfectly type the entire data structure AND propose a process to do perform version updates to sync client/server are going to lose most of the time. The zero cost of starting with JSON is too com…

> When judging which alternative will succeed, lower perceived human cost beats lower machine cost every time. Yup this is it. No architect considers using protos unless there is an explicit need for it. And the explicit need is most times using gRPC. Unless the alternative allows for zero cost startup and debugging by just doing `console.log()`, they won't replace JSON any time soon. Edit: Just for context, I'm not…

Print debugging is fine and all but I find that it pays massive dividends to learn how to use a debugger and actually inspect the values in scope rather than guessing which are worth printing. It also is useless when you need to debug a currently running system and can't change the code.

And since you need to translate it anyway, there's not much benefit in my mind to using something like msgpack which is more compact and self describing, you just need a decoder to convert to json when you display it.

Re: Why I stopped using JSON for my APIs

#86

Post by EdwardDiego »

recursivecaveat wrote: post #54

My dream binary format is schema driven, as compact and efficient as Capt Proto or such, but just optionally embeds the entire schema into the message. Then we can write a vim plugin that just opens the file in human readable form without having to fish for the schema. Whenever I am using binary formats, it's because I have a list of millions of objects of the same types. Seems to me that you may as well tack 1KB of…

As another user suggested, Avro is something to look into.

Re: Why I stopped using JSON for my APIs

#87

Post by PantaloonFlames »

PunchyHamster wrote: post #80

"I've used this optimization technique to make app faster" The app 20req/sec The app after optimizations: 20req/sec (It waits for db query anyway)

Yes. Proto makes sense when the request rate is much higher and the network is constrained.

Otherwise, json is sufficient.

Re: Why I stopped using JSON for my APIs

#88

Post by __s »

jonny_eh wrote: post #72
JoshMock wrote: post #64

Protobuf is a great format with a lot of benefits, but it's missing one that I wish it could support: zero-copy. The ability to transport data between processes, services and languages with effectively zero time spent on serialization and deserialization. It appears possible in some cases but it's not universally the case. Which means that similar binary transport formats that do support zero-copy, like Cap'n Proto,…

Is that a format/serialization issue, or library/implementation issue?

Format: https://news.ycombinator.com/item?id=23589117

Re: Why I stopped using JSON for my APIs

#89

Post by PantaloonFlames »

Jemaclus wrote: post #11

"Better than JSON" is a pretty bold claim, and even though the article makes some great cases, the author is making some trade-offs that I wouldn't make, based on my 20+ year career and experience. The author makes a statement at the beginning: "I find it surprising that JSON is so omnipresent when there are far more efficient alternatives." We might disagree on what "efficient" means. OP is focusing on computer effi…

Yes to all of this.

Also the “us” is ever-changing in a large enough system. There are always people joining and leaving the team. Always, many people are approximately new, and JSON lets them discover more easily.

Post reply on HN