Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I stopped using JSON for my APIs

aloisdeniel.com

Post reply on HN
Print view View original

61–70 of 245 posts

Re: Why I stopped using JSON for my APIs

#61

Post by teleforce »

Kudos to the poster and the author of this article. I think this is by far the most insightful technical post I've read this year on HN.

>If you develop or use an API, there’s a 99% chance it exchanges data encoded in JSON.

Just wondering if the inherent defiencies of JSON can somewhat be improved by CUE lang since the former is very much pervasive and the latter understand JSON [1],[2].

[1] Configure Unify Execute (CUE): Validate, define, and use dynamic and text‑based data:

https://cuelang.org/

[2] Cue – A language for defining, generating, and validating data:

https://news.ycombinator.com/item?id=20847943

Re: Why I stopped using JSON for my APIs

#62

Post by koakuma-chan »

dhussoe wrote: post #47
brabel wrote: post #17

Earlier quoted context omitted.

No type system survives going through a network.

yes, but any sane JSON parsing library (Rust Serde, kotlinx-serialization, Swift, etc.) will raise an error when you have the wrong type or are missing a required field. and any JSON parsing callsite is very likely also an IO callsite so you need to handle errors there anyways, all IO can fail. then you log it or recover or whatever you do when IO fails in some other way in that situation. this seems like a problem o…

> strictly parsing (parse, don't validate)

That's called validating? Zod is a validation library.

But yeah, people really need to start strictly parsing/validating their data. One time I had an interview and I was told yOu DoN'T tRuSt YoUr BaCkeNd?!?!?!?

Re: Why I stopped using JSON for my APIs

#64

Post by JoshMock »

Protobuf is a great format with a lot of benefits, but it's missing one that I wish it could support: zero-copy. The ability to transport data between processes, services and languages with effectively zero time spent on serialization and deserialization.

It appears possible in some cases but it's not universally the case. Which means that similar binary transport formats that do support zero-copy, like Cap'n Proto, offer most or all of the perks described in this post, with the addition of ensuring that serialization and deserialization are not a bottleneck when passing data between processes.

Re: Why I stopped using JSON for my APIs

#65

Post by morshu9001 »

zzo38computer wrote: post #58
morshu9001 wrote: post #52

Earlier quoted context omitted.

In what ASN1 application is protobuf spec too limited? I've used protobuf for tons of different things, it's always felt right. Though I understand certain encodings of ASN1 can have better performance for specific things.

Numbers bigger than 64-bits, character sets other than Unicode (and ASCII), OIDs, etc.

These are only scalars that you'd encode into bytes. I guess it's slightly annoying that both ends have to agree on how to serialize rather than protobuf itself doing it, but it's not a big enough problem.

Also I don't see special ASN1 support for non-Unicode string encodings, only subsets of Unicode like ascii or printable ascii. It's a big can of worms once you bring in things like Latin-1.

Re: Why I stopped using JSON for my APIs

#67

Post by dhussoe »

koakuma-chan wrote: post #62
dhussoe wrote: post #47

Earlier quoted context omitted.

yes, but any sane JSON parsing library (Rust Serde, kotlinx-serialization, Swift, etc.) will raise an error when you have the wrong type or are missing a required field. and any JSON parsing callsite is very likely also an IO callsite so you need to handle errors there anyways, all IO can fail. then you log it or recover or whatever you do when IO fails in some other way in that situation. this seems like a problem o…

> strictly parsing (parse, don't validate) That's called validating? Zod is a validation library. But yeah, people really need to start strictly parsing/validating their data. One time I had an interview and I was told yOu DoN'T tRuSt YoUr BaCkeNd?!?!?!?

"parse don't validate" is from: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...

looking at zod (assuming https://zod.dev) it is a parsing library by that definition — which isn't, like, an official definition or anything, one person on the internet came up with it, but I think it is good at getting the principle across

under these definitions a "parser" takes some input and returns either some valid output (generally a more specific type, like String -> URL) or an error, whereas a "validator" just takes that input and returns a boolean or throws an error or whatever makes sense in the language.

eta: probably part of the distinction here is that since zod is a JS library the actual implementation can be a "validator" and then the original parsed JSON input can just be returned with a different type. "parse don't validate" is (IMO) more popular in languages like Rust where you would already need to parse the JSON to a language-native structure from the original bytes, or to some "JSON" type like https://docs.rs/serde_json/latest/serde_json/enum.Value.html that are generally awkward for application code (nudging you onto the happy parsing path).

Re: Why I stopped using JSON for my APIs

#68

Post by dfabulich »

> With JSON, you often send ambiguous or non-guaranteed data. You may encounter a missing field, an incorrect type, a typo in a key, or simply an undocumented structure. With Protobuf, that’s impossible. Everything starts with a .proto file that defines the structure of messages precisely.

This deeply misunderstands the philosophy of Protobuf. proto3 doesn't even support required fields. https://protobuf.dev/best-practices/dos-donts/

> Never add a required field, instead add `// required` to document the API contract. Required fields are considered harmful by so many they were removed from proto3 completely.

Protobuf clients need to be written defensively, just like JSON API clients.

Re: Why I stopped using JSON for my APIs

#69

Post by phyzome »

spagoop wrote: post #2

Is it just me or is this article insanely confusing? With all due respect to the author, please be mindful of copy editing LLM-assisted writing. There is a really interesting discussion underneath of this as to the limitations of JSON along with potential alternatives, but I can't help but distrust this writing due to how much it sounds like an LLM.

It wasn't confusing, but yeah, it smelled strongly of LLMs.

Re: Why I stopped using JSON for my APIs

#70

Post by klysm »

dfabulich wrote: post #68

> With JSON, you often send ambiguous or non-guaranteed data. You may encounter a missing field, an incorrect type, a typo in a key, or simply an undocumented structure. With Protobuf, that’s impossible. Everything starts with a .proto file that defines the structure of messages precisely. This deeply misunderstands the philosophy of Protobuf. proto3 doesn't even support required fields. https://protobuf.dev/best-pra…

It’s also conflating the serialization format with contracts
Post reply on HN