Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I stopped using JSON for my APIs

aloisdeniel.com

Post reply on HN
Print view View original

191–200 of 245 posts

Re: Why I stopped using JSON for my APIs

#191

Post by almosthere »

MobiusHorizons wrote: post #101
almosthere wrote: post #92

Earlier quoted context omitted.

why are you against static types in languages? I once converted a fairly large JS codebase to TS and I found about 200 mismatching names/properties all over the place. Tons of properties we had nulls suddenly started getting values.

Sounds like this introduced behavior changes. How did you evaluate if the new behavior was desirable or not? I’ve definitely run into cases where the missing fields were load bearing in ways the types would not suggest, so I never take it for granted that type error in prod code = bug

it was desirable because our reason for the conversion was subtle bugs all over the place where data was disappearing.

Re: Why I stopped using JSON for my APIs

#192

Post by duped »

mekoka wrote: post #177
duped wrote: post #170

Earlier quoted context omitted.

That's just an educated guess. You can also do it with a debugger.

The debugger is fine, but it's not the key to unlock some secret skill level that you make it out to be. https://lemire.me/blog/2016/06/21/i-do-not-use-a-debugger/

I didn't say it's some arcane skill, just that it's a useful one. I would also agree that _reading the code_ to find a bug is the most useful debugging tool. Debuggers are second. Print debugging third.

And that lines up with some of the appeals to authority there that are good, and that are bad (edited to be less toxic)

Re: Why I stopped using JSON for my APIs

#193

Post by orphea »

davedx wrote: post #124

"Ultra-efficient" Searched the article, no mention of gzip, and how most of the time all that text data (html, js and css too!) you're sending over the wire will be automatically compressed to...... an efficient binary format! So really, the author should compare protobufs to gzipped JSON

Do you compress data if it's then encrypted (HTTPS)?

https://en.wikipedia.org/wiki/CRIME

Re: Why I stopped using JSON for my APIs

#194

Post by theamk »

themafia wrote: post #125
theamk wrote: post #81

Earlier quoted context omitted.

ASN.1 has too much stuff. The moment you write "I made ASN.1 decoder/encoder", someone will throw TeletexString or BMPString at it. Or inheritance, as morshu9001 sad. So at this point: - You can support all those features, and your ASM.1 library will be horribly bloated and over-engineered. - You can support your favorite subset, but then you cannot say it's ASN.1 anymore. It will be "ASN.brabel", which only has one…

> someone will throw TeletexString or BMPString ASCII with escapes and UCS-2. > horribly bloated and over-engineered. It's no more or less complicated than XML, JSON or CSV. Which is why you can use ASN.1 to serialize to and from all these formats. ASN.1 provides you an additional level of schema above these. It simply allows you to describe your problem. I find ASN.1 far more sane and useful than something like JSON…

Nope, TeletexString is ITU T.61, a.k.a codepage 1036. So Backspace (0x08) is OK, but Tab (0x09) is not.

What, your implementation does not include CP1036 to Unicode translation table? Sorry, it's no longer ASN.1, it's now ASN.themafia.

Oh, it does? Then how about xmlhstring vs hstring, are you handing difference properly?

What about REAL type? Does your asn.1 library include support for arbitrary-precision floating point numbers, both base 2 and 10? And no, you cannot use "double", I am sure there is an application out there which uses base 10 reals, or has 128 bits of mantissa.

ASN.1 is full of overengineered, ancient things like those, and the worst part - once you actually start using it to interoperate with other software, there is a good chance you'll see them. If you want something that people actually implement fully, choose something else.

Re: Why I stopped using JSON for my APIs

#195

Post by IshKebab »

brabel wrote: post #15

Mandatory comment about ASN.1, a protocol from 1984, already did what Protobuf does, with more flexibility. Yes, it's a bit ugly but if you stick to the DER encoding it's really not worse than Protbuf at all. Check out the Wikipedia example: https://en.wikipedia.org/wiki/ASN.1#Example_encoded_in_DER Protobuf is ok but if you actually look at how the serializers work, it's just too complex for what it achieves.

I don't see how you can seriously criticise Protobuf's very simple encoding scheme as being too complex while recommending ASN.1!!

Totally mad.

Re: Why I stopped using JSON for my APIs

#196

Post by morshu9001 »

cryptonector wrote: post #111
zzo38computer wrote: post #78

Earlier quoted context omitted.

ASN.1 has support for ISO 2022 as well as ASCII and Unicode (ASCII is a subset of Unicode as well as a subset of ISO 2022). (My own nonstandard extensions add a few more (such as TRON character code and packed BCD), and the standard unrestricted character string type can be used if you really need arbitrary character sets.) (Unicode is not a very good character set, anyways.) Also, DER allows to indicate the type of…

To be fair, if you don't need to support anything other than Unicode, then this is not an advantage, and over time we're all going to need non-Unicode less and less. That said I'm a big fan of ASN.1 (see my comment history).

I'm still confused how these ISO 2022 strings even work, and the ASN1 docs discourage using the UniversalString and GraphicString types. All these different string types are intimidating if I just want unicode/ascii, and even if I were using an obscure encoding, I'd use generic bytes instead of wanting asn1 to care about it.

Re: Why I stopped using JSON for my APIs

#197

Post by cryptonector »

whatevaa wrote: post #136
cryptonector wrote: post #114

Earlier quoted context omitted.

That's not ASN.1's fault though.

Json doesn't support comments specifically to not allow parsing directives, that means less customization. More customization of interoperability protocols is not always a good thing.

The compiler I [occasionally] work on does not abuse comments for directives. All directives in that compiler are out of band.

Re: Why I stopped using JSON for my APIs

#198

Post by cryptonector »

whatevaa wrote: post #137
cryptonector wrote: post #112

Earlier quoted context omitted.

I've been working on and with Kerberos and PKIX for decades. I don't find ASN.1 to be a problem as long as you have good tooling or are willing to build it. The specs are a pleasure to read -- clear, concise, precise, and approachable (once you have a mental model for it anyways). Of course, I am an ASN.1 compiler maintainer, but hey, I had to become one because the compiler I was using was awesome but not good enoug…

You just said it. You had to become compiler maintainer to make it good enough.

Here's the problem though: people have used the absence of tooling to justify the creation of new, supposedly-superior schemas and codecs that by definition have strictly less tooling available on day zero and which invariably turn out to be worse than ASN.1/DER were in 1984 because the authors also refused to study the literature to see what good ideas they could pick up. That's how we end up with:

- PB being a TLV encoding, just like DER, with all the same problems

   (Instead PB should have been inspired by XDR or OER, but not DER.)

 - PB's IDL requiring explicitly tagging every field of every data structure(!) even though ASN.1 never required tagging every field, and even though ASN.1 eventually adopted automatic tagging.

 - PB's very naive approach to extensibility that is just like 1984 ASN.1's.
It's a mistake.

Some people, when faced with a dearth of tooling, will write said tooling. Other people will say that the technology in question is a nightmare, and some of those people will then go on to invent a worse wheel.

I'd be ecstatic to use something other than ASN.1 if it wasn't a poor reinvention of it.

Re: Why I stopped using JSON for my APIs

#199

Post by cryptonector »

morshu9001 wrote: post #196
cryptonector wrote: post #111

Earlier quoted context omitted.

To be fair, if you don't need to support anything other than Unicode, then this is not an advantage, and over time we're all going to need non-Unicode less and less. That said I'm a big fan of ASN.1 (see my comment history).

I'm still confused how these ISO 2022 strings even work, and the ASN1 docs discourage using the UniversalString and GraphicString types. All these different string types are intimidating if I just want unicode/ascii, and even if I were using an obscure encoding, I'd use generic bytes instead of wanting asn1 to care about it.

GeneralString relies on control characters to "load" character sets into the C0 and C1 registers. This is madness -- specifically it's pre-Unicode madness, but before Unicode it made sense.

Re: Why I stopped using JSON for my APIs

#200

Post by IshKebab »

barremian wrote: post #53
written-beyond wrote: post #5

Idk I built a production system and ensured all data transfers, client to server and server to client were proto buf and it was a pain. Technically, it sounds really good but the actual act of managing it is hell. That or I need a lot of practice to use them, at that point shouldn't I just use JSON and get on with my life.

I think the it-just-works nature and human readability for debugging JSON cannot be overstated. Most people and projects are content to just use JSON even if protos offer some advantages, if not only to save time and resources. Whether the team saves times in the longer when using protos is a question in its own.

There are plenty of it-doesn't-just-work things about JSON though. Sending binary data or 64-bit integers is a huge pain. Or maps with non-string keys, or ordered maps. Plus JSON doesn't scale well with message size because it doesn't use TLV so parsing any of a message requires parsing all of it.

It's not some perfect format.

That said, I'm disappointed with Protobuf too. Especially the handling of optional/default fields. I believe they did eventually add an `optional` tag so you can at least distinguish missing vs default field values.

The lack of required fields makes it very annoying to work with though. And no, there's no issue with required fields in general. The only reason it doesn't have them is because the implementation in Protobuf 2 caused issues and Google had to have a smooth transition from that.

If you're starting a greenfield project it seems silly to opt in to Google's tech debt.

If you're thinking "but schema evolution!!" well yeah all you need is a way to have versioned structs and then you can mark fields as being required for ranges of versions. So you can still remove required fields without breaking backwards compatibility.

Post reply on HN