Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I stopped using JSON for my APIs

aloisdeniel.com

Post reply on HN
Print view View original

111–120 of 245 posts

Re: Why I stopped using JSON for my APIs

#111

Post by cryptonector »

zzo38computer wrote: post #78
morshu9001 wrote: post #65

Earlier quoted context omitted.

These are only scalars that you'd encode into bytes. I guess it's slightly annoying that both ends have to agree on how to serialize rather than protobuf itself doing it, but it's not a big enough problem. Also I don't see special ASN1 support for non-Unicode string encodings, only subsets of Unicode like ascii or printable ascii. It's a big can of worms once you bring in things like Latin-1.

ASN.1 has support for ISO 2022 as well as ASCII and Unicode (ASCII is a subset of Unicode as well as a subset of ISO 2022). (My own nonstandard extensions add a few more (such as TRON character code and packed BCD), and the standard unrestricted character string type can be used if you really need arbitrary character sets.) (Unicode is not a very good character set, anyways.) Also, DER allows to indicate the type of…

To be fair, if you don't need to support anything other than Unicode, then this is not an advantage, and over time we're all going to need non-Unicode less and less. That said I'm a big fan of ASN.1 (see my comment history).

Re: Why I stopped using JSON for my APIs

#112

Post by cryptonector »

morshu9001 wrote: post #37
pphysch wrote: post #26

Earlier quoted context omitted.

> ASN.1, a protocol from 1984, already did what Protobuf does, with more flexibility. After working heavily with SNMP across a wide variety of OEMs, this flexibility becomes a downside. Or SNMP/MIBs were specified at the wrong abstraction level, where the ASN.1 flexibility gives mfgs too much power to do insane and unconventional things.

Yeah same, ASN.1 was a nightmare when I was dealing with LTE

I've been working on and with Kerberos and PKIX for decades. I don't find ASN.1 to be a problem as long as you have good tooling or are willing to build it. The specs are a pleasure to read -- clear, concise, precise, and approachable (once you have a mental model for it anyways).

Of course, I am an ASN.1 compiler maintainer, but hey, I had to become one because the compiler I was using was awesome but not good enough, so I made it good enough.

I'm curious what made it a nightmare for you.

Re: Why I stopped using JSON for my APIs

#113

Post by taftster »

wg0 wrote: post #45

Protos don't work out of the box in any browsers as far as I checked last time unless you're willing to deploy a proxy in front to do the translation and it requires extra dependency on the browser as well. Plus - tooling. JSON might not be simpler or strict but it gets the job done. If JSON's size or performance is causing you to go out of business, you surely have bigger problems than JSON.

Right, I find the use of protobuf lacking with direct support in the browser. Since JSON is a native data encoding format of the browser (effectively), it's just easier to have a JSON-based API.

Yes, there are abstractions or other hacks that would bolt on protobuf support in the browser, but that's not ideal in my mind.

Protobuf is an ideal exchange format when you're not dealing with the public as a whole. That is, a private or corporate API for your data processing pipelines, etc.

Re: Why I stopped using JSON for my APIs

#114

Post by cryptonector »

pphysch wrote: post #26
brabel wrote: post #15

Mandatory comment about ASN.1, a protocol from 1984, already did what Protobuf does, with more flexibility. Yes, it's a bit ugly but if you stick to the DER encoding it's really not worse than Protbuf at all. Check out the Wikipedia example: https://en.wikipedia.org/wiki/ASN.1#Example_encoded_in_DER Protobuf is ok but if you actually look at how the serializers work, it's just too complex for what it achieves.

> ASN.1, a protocol from 1984, already did what Protobuf does, with more flexibility. After working heavily with SNMP across a wide variety of OEMs, this flexibility becomes a downside. Or SNMP/MIBs were specified at the wrong abstraction level, where the ASN.1 flexibility gives mfgs too much power to do insane and unconventional things.

That's not ASN.1's fault though.

Re: Why I stopped using JSON for my APIs

#115

Post by lenkite »

There is no mention in this thread of how the author is using Dart and Shelf for his REST API's. Code is rather readable and elegant. This is not a combo I have every tried before. Does anyone have any experience of how it compares versus REST services written in Go/Rust/Python ?

Re: Why I stopped using JSON for my APIs

#117

Post by masklinn »

Seattle3503 wrote: post #100
shortrounddev2 wrote: post #83

Earlier quoted context omitted.

Most web frameworks do both at the same time to the point where having to write code which enforced a type contract after deserializing is a delabreaker for me. I eant to be able to define my DTOs in one place, once, and have it both deserialize and enforce types/format. Anything else is code smell

I'm in the same boat. I mostly write Rust and Python. Using serde_json and Pydantic, you get deserialization and validation at the same time. It allows you to de-serialize really "tight" types. Most of my APIs are internal APIs that accept breaking changes easily. My experience with protobufs is that it was created to solve problems in large systems with many teams and APIs, where backwards compatibility is important…

> My experience with protobufs is that it was created to solve problems in large systems with many teams and APIs

Also significant distribution such that it’s impossible to ensure every system is updated in lockstep (at least not without significant downtime), and high tail latencies e.g. a message could be stashed into a queue or database and processed hours or days later.

Re: Why I stopped using JSON for my APIs

#118

Post by cookiengineer »

dfabulich wrote: post #68

> With JSON, you often send ambiguous or non-guaranteed data. You may encounter a missing field, an incorrect type, a typo in a key, or simply an undocumented structure. With Protobuf, that’s impossible. Everything starts with a .proto file that defines the structure of messages precisely. This deeply misunderstands the philosophy of Protobuf. proto3 doesn't even support required fields. https://protobuf.dev/best-pra…

Isn't the core issue just language and implementation differences of clients vs servers here?

I went all in with Go's Marshalling concept, and am using my Gooey framework on the client side nowadays. If you can come around Go's language limitations, it's pretty nice to use and _very_ typesafe. Just make sure to json:"-" the private fields so they can't be injected.

[1] shameless drop https://github.com/cookiengineer/gooey

Re: Why I stopped using JSON for my APIs

#119

Post by rock_artist »

I feel few points weren’t addressed in the article.

1. Size, biggest problem with JSON can happen when things gets too big. So here other formats might be better. Yet, as a reminder JSON has the binary version named BSON.

2. Zero batteries. JSON is readable by humans but also almost self explanatory format. Most languages has built in or quick drop in for json. Still, it’s easy to implement a limited JSON parser from scratch when in need (eg. Pure on func in C on a tiny device).

Working with Protobuf and MsgPack in the past, You have much more tooling involved especially if data passes between parts written in different languages.

3. Validation, JSON is simple. But there are solutions such as JSON Schema.

Re: Why I stopped using JSON for my APIs

#120

Post by paulddraper »

rock_artist wrote: post #119

I feel few points weren’t addressed in the article. 1. Size, biggest problem with JSON can happen when things gets too big. So here other formats might be better. Yet, as a reminder JSON has the binary version named BSON. 2. Zero batteries. JSON is readable by humans but also almost self explanatory format. Most languages has built in or quick drop in for json. Still, it’s easy to implement a limited JSON parser from…

BSON is not simply a binary encoding of JSON.

It is a JSON superset with binary encoding, created by MongoDB. (And there is even a JSON encoding of BSON, called extended JSON.)

AFAIK there is no widely adopted binary pure adaptation of JSON. (There are application-specific storage formats, like PostgreSQL JSONB, or SQLite JSONB.)

——-

Moreover, JSON is relatively compact. BSON or other self descriptive binary formats are often around the same size of JSON. MessagePack aggressively tries to be compact and is, depending on the data. BSON doesn’t try to be compact, rather it improves the parse speed.

Post reply on HN