Live data from Hacker News

Show HN: Skir – like Protocol Buffer but better

skir.build

41–50 of 69 posts

Re: Show HN: Skir – like Protocol Buffer but better

#41

That “compact JSON” format reminds me if the special protobufs JSON format that Google uses in their APIs that has very little public documentation. Does anyone happen to know why Google uses that, and to OP, were you inspired by that format?

I don't know but if I had to guess.

1. Google uses protobufs everywhere, so having something that behaves equivalently is very valuable. For example in protobuf renaming fields is safe, so if they used field names in the JSON it would be protobuf incompatible.

2. It is usually more efficient because you don't send field names. (Unless the struct is very sparse it is probably smaller on the wire, serialized JS usage may be harder to evaluate since JS engines are probably more optimized for structs than heterogeneous arrays).

3. Presumably the ability to use the native JSON parsing is beneficial over a binary parser in many cases (smaller code size and probably faster until the code gets very hot and JITed).

Re: Show HN: Skir – like Protocol Buffer but better

#42
> For optional types, 0 is decoded as the default value of the underlying type (e.g. string? decodes 0 as "", not null).

In the "dense JSON" format, isn't representing removed/absent struct fields with `0` and not `null` backwards incompatible?

If you remove or are unaware of a `int32?` field, old consumers will suddenly think the value is present as a "default" value rather than absent

Re: Show HN: Skir – like Protocol Buffer but better

#43
post #42

> For optional types, 0 is decoded as the default value of the underlying type (e.g. string? decodes 0 as "", not null). In the "dense JSON" format, isn't representing removed/absent struct fields with `0` and not `null` backwards incompatible? If you remove or are unaware of a `int32?` field, old consumers will suddenly think the value is present as a "default" value rather than absent

That is correct and that is a good catch, the idea though is that when you remove a field you typically do that after having made sure that all code no longer read from the removed field and that all binaries have been deployed.

Re: Show HN: Skir – like Protocol Buffer but better

#44
post #43
post #42

> For optional types, 0 is decoded as the default value of the underlying type (e.g. string? decodes 0 as "", not null). In the "dense JSON" format, isn't representing removed/absent struct fields with `0` and not `null` backwards incompatible? If you remove or are unaware of a `int32?` field, old consumers will suddenly think the value is present as a "default" value rather than absent

That is correct and that is a good catch, the idea though is that when you remove a field you typically do that after having made sure that all code no longer read from the removed field and that all binaries have been deployed.

How does this work if, for example, you persist the data in a database?

Re: Show HN: Skir – like Protocol Buffer but better

#47

That “compact JSON” format reminds me if the special protobufs JSON format that Google uses in their APIs that has very little public documentation. Does anyone happen to know why Google uses that, and to OP, were you inspired by that format?

I don't know the reason TextFormat was invented, but in practice it's way easier to work with TextFormat than JSON in the context of Protos.

Consider numeric types -

JSON: number aka 64-bit IEEE 754 floating point

Proto: signed and unsigned int 8, 16, 32, 64-bit, float, double

I can only imagine the carnage saved by not accidentally chopping of the top 10 bits (or something similar) of every int64 identifier when it happens to get processed by a perfectly normal, standards compliant JSON processor.

It's true that most int64 fields could be just fine with int54. It's also true that some fields actually use those bits in practice.

Also, the JSPB format references tag numbers rather than field names. It's not really readable. For TextProto it might be a log output, or a config file, or a test, which are all have ways of catching field name discrepancies (or it doesn't matter). For the primary transport layer to the browser, the field name isn't a forward compatible/safe way to reference the schema.

So oddly the engineers complaining about the multiple text formats are also saved from a fair number of bugs by being forced to use tools more suited to their specific situation.

Re: Show HN: Skir – like Protocol Buffer but better

#50
This seems a Chesterton's fence fail.

protobuf solved serialization with schema evolution back/forward compatibility.

Skir seems to have great devex for the codegen part, but that's the least interesting aspect of protobufs. I don't see how the serialization this proposes fixes it without the numerical tagging equivalent.

Post reply on HN