Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Critique of the Cap'n Proto Schema Language (2019)

zenhack.net

Post reply on HN
Print view View original

41–50 of 73 posts

Re: A Critique of the Cap'n Proto Schema Language (2019)

#41

Post by jffhn »

>Cap’n Proto unions are not first class types. Instead, they are fields of structs

In some binary data description format, I didn't include unions because they are just a special case of what I called "conditional sequences". But then I added them, because it is very difficult or impossible to figure out whether a set of conditional sequences corresponds to a union (all must have same size, which is only known at encoding/decoding, and only one must be present), which you need to know when converting into more restrictive formats that only support unions.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#42

Post by afiori »

signa11 wrote: post #30
afiori wrote: post #18

Earlier quoted context omitted.

it depends on what type of safety. The schema language might for example allow you to specify that an input string/blob should be smaller than 10MB and refuse to deserialize it if it is longer, same for array/list/vector length.

> ... allow you to specify that an input string/blob should be smaller than 10MB and refuse to deserialize it if it is longer ... why ? are there no cases where serializing even larger file is valid ?

sure, a lot of cases, I suspect that S3 upload limits are different from imgur.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#43

Post by afiori »

seangrogg wrote: post #23
afiori wrote: post #18

Earlier quoted context omitted.

it depends on what type of safety. The schema language might for example allow you to specify that an input string/blob should be smaller than 10MB and refuse to deserialize it if it is longer, same for array/list/vector length.

It feels like a check against an input size of 10MB is something you would do well before deserialization, no?

not if it is a message you receive from a third party.

A concrete example might be a batching third party client: the app sends N messages in a single batch and each message has its own size limit.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#44

Post by posix86 »

klabb3 wrote: post #11

Great post, digestible without experience in capn proto specifics. I really like that they talk in terms of “features carrying their own weight” and survey real world code to see what cases features solve for, and how (surprisingly little) certain features are actually used. A side effect of dropping features when designing for code generation (in this case) is that it makes things more concise for everyone else as w…

Premature optimization is the root of all evil - again. Applies to performance and features. Unless you can proove (not formally, but for yourself) that feature will make things better, don't implement it.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#45

Post by naasking »

jffhn wrote: post #41

>Cap’n Proto unions are not first class types. Instead, they are fields of structs In some binary data description format, I didn't include unions because they are just a special case of what I called "conditional sequences". But then I added them, because it is very difficult or impossible to figure out whether a set of conditional sequences corresponds to a union (all must have same size, which is only known at enc…

Indeed, there's a good reason why sums (unions) and products (records) are the most primitive abstractions in various logics and mathematics.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#46

Post by tom_ »

seangrogg wrote: post #23
afiori wrote: post #18

Earlier quoted context omitted.

it depends on what type of safety. The schema language might for example allow you to specify that an input string/blob should be smaller than 10MB and refuse to deserialize it if it is longer, same for array/list/vector length.

It feels like a check against an input size of 10MB is something you would do well before deserialization, no?

The limit might apply to some specific part of the message, rather than the whole. You can't check this without actually deserialising, or at least doing most of the same work.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#47

Post by tom_ »

muxator wrote: post #17
hlieberman wrote: post #4

Rest in peace, Ian, my friend.

For context: it seems that the author passed away on last July https://www.winchesteruu.org/2023/07/25/joys-sorrows-and-tra...

"Last" in this context refers to the year. Last July would be July 2022; July 2023 is this July. July 2023 will become last July at the start of 2024.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#48

Post by eviks »

jiggawatts wrote: post #40
signa11 wrote: post #29

Earlier quoted context omitted.

> Serialization and parsing are security minefields and it is dangerously naive to just hand-wave that away. well, i am not hand-waving them away, i am not sure what can the serialization framework possibly _do_ to make things secure during the serialization ? when execution of user-supplied code is allowed (in the examples that you have outlined above), surely, the layer _executing_ the code cannot really do anythin…

> I am not sure what can the serialization framework possibly _do_ to make things secure during the serialization Loads of things! A strict specification that can only be interpreted one way goes very far. E.g.: a machine-readable BNF grammar file or something similar with no ambiguities. A conformance test suite covering corner-cases is surprisingly effective, even with a supposedly perfect spec. "Be strict with wha…

what's the best modern alternative that is designed in this way?

Re: A Critique of the Cap'n Proto Schema Language (2019)

#50

Post by kentonv »

umvi wrote: post #49

Why do all these serialization frameworks like protobufs, flatbuffers, capnproto, etc. have bespoke schema languages? Why not just use JSON-based schema so that you don't need custom parsers for it? It would definitely make metaprogramming easier as well.

Because JSON-based schemas are excessively verbose making them painful to read and write, to the point where really nobody wants to do so.

Consider:

    struct Person {
      name @0 :Text;
      age @1 :UInt16;
    }
vs.:

    "declarations": [
      {
        "name": "Person",
        "kind": "struct",
        "fields": [
          {
            "name": "name",
            "type": "text",
            "ordinal": 0
          }, {
            "name": "age",
            "type": "uint16",
            "ordinal": 1
          }
        ]
      }
    ]
Post reply on HN