Live data from Hacker News

A Critique of the Cap'n Proto Schema Language (2019)

zenhack.net

51–60 of 73 posts

Re: A Critique of the Cap'n Proto Schema Language (2019)

#51
post #49

Why do all these serialization frameworks like protobufs, flatbuffers, capnproto, etc. have bespoke schema languages? Why not just use JSON-based schema so that you don't need custom parsers for it? It would definitely make metaprogramming easier as well.

Avro uses json for its schema language.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#52
post #50
post #49

Why do all these serialization frameworks like protobufs, flatbuffers, capnproto, etc. have bespoke schema languages? Why not just use JSON-based schema so that you don't need custom parsers for it? It would definitely make metaprogramming easier as well.

Because JSON-based schemas are excessively verbose making them painful to read and write, to the point where really nobody wants to do so. Consider: struct Person { name @0 :Text; age @1 :UInt16; } vs.: "declarations": [ { "name": "Person", "kind": "struct", "fields": [ { "name": "name", "type": "text", "ordinal": 0 }, { "name": "age", "type": "uint16", "ordinal": 1 } ] } ]

Also JSON schemas can represent the exact same type in multiple ways, meaning some form of normalization is required prior.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#53
post #42
post #30

Earlier quoted context omitted.

> ... allow you to specify that an input string/blob should be smaller than 10MB and refuse to deserialize it if it is longer ... why ? are there no cases where serializing even larger file is valid ?

sure, a lot of cases, I suspect that S3 upload limits are different from imgur.

and feel free to do that in _your_ application. don’t shackle others with the limitations of your domain.

mechanism vs policy and all that.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#54
post #41

>Cap’n Proto unions are not first class types. Instead, they are fields of structs In some binary data description format, I didn't include unions because they are just a special case of what I called "conditional sequences". But then I added them, because it is very difficult or impossible to figure out whether a set of conditional sequences corresponds to a union (all must have same size, which is only known at enc…

Is there any serialization format that handles (discriminated) unions well?

Re: A Critique of the Cap'n Proto Schema Language (2019)

#55
post #8
post #3

Earlier quoted context omitted.

yes ?

Knowing that it's from a few years ago gives helpful information to the potential reader. 1) Someone may have read the essay before, remembered the point, but forgotten the exact title. It has, after all, been posted on HN twice before. Simply reading the comment that it's from 4 years ago may be enough to confirm that it's the same article, as it did with me. While the essay has the date right at the top, in general…

Yeah, that ^^^^ was pretty much my reasoning. Thanks for articulating it so clearly.

Honestly I didn't expect my original comment to be controversial.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#56
post #8

Earlier quoted context omitted.

Knowing that it's from a few years ago gives helpful information to the potential reader. 1) Someone may have read the essay before, remembered the point, but forgotten the exact title. It has, after all, been posted on HN twice before. Simply reading the comment that it's from 4 years ago may be enough to confirm that it's the same article, as it did with me. While the essay has the date right at the top, in general…

The submission guidelines contain no such allowance. They specify a very limited set of cases where the title of a submission can be modified from the web page title.

Thanks. I wasn't aware that putting a date in the title might violate the guidelines.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#57

Earlier quoted context omitted.

The submission guidelines contain no such allowance. They specify a very limited set of cases where the title of a submission can be modified from the web page title.

Thanks. I wasn't aware that putting a date in the title might violate the guidelines.

It does not.

You can see dang, one of the moderators, write: "It's the convention on HN to add the year to a title when the article isn't from this year." at https://news.ycombinator.com/item?id=27867435 ,

write "On HN, it's the convention to add the year when a story is more than a year and a bit old." at https://news.ycombinator.com/item?id=16306830

and write "It's just the convention to add the year in parens (and for commenters to mention when we missed one)." at https://news.ycombinator.com/item?id=8244765 .

Found via https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... .

If orhmeh09 has issue with how to interpret the submission guidelines, I suggest contacting hn@ycombinator.com to resolve the issue.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#58
post #53
post #42

Earlier quoted context omitted.

sure, a lot of cases, I suspect that S3 upload limits are different from imgur.

and feel free to do that in _your_ application. don’t shackle others with the limitations of your domain. mechanism vs policy and all that.

I believe I have already justified why it might be useful at the protocol/schema level in ways that cannot be replicated at the application level: to eagerly fail on expensive (eg memory) deserialization.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#59
post #7

Earlier quoted context omitted.

It's common on HN to note the date for reposts. The mods will usually add the year to the title if it's not included. So that's probably all that CoastalCoder was signaling.

Users are not advised to do this themselves. The submission guidelines contain no such allowance. They specify a very limited set of cases where the title of a submission can be modified from the web page title.

This is a long-established convention to include the year if the article is a year and a bit old.

See dang, one of the moderators, affirm this multiple times at https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... , most recently 17 days ago.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#60
post #51
post #49

Why do all these serialization frameworks like protobufs, flatbuffers, capnproto, etc. have bespoke schema languages? Why not just use JSON-based schema so that you don't need custom parsers for it? It would definitely make metaprogramming easier as well.

Avro uses json for its schema language.

But also has a schema language, and a avro.reflect which can construct schema from java class.

Avro schema in Json is very unreadable in my experience.

Another bad thing is something like ["int", "null"] gets formatted over 2 lines by standard JSON formatters, making it even harder to read.

Post reply on HN