A Critique of the Cap'n Proto Schema Language (2019) • Hacker News Board

Re: A Critique of the Cap'n Proto Schema Language (2019)

#31

Post by bsder » Sun, Aug 20, 2023, 9:30 AM UTC

woodruffw wrote: post #14

ardel95 wrote: post #5

I think one of the most desirable and under-appreciated goals of schema languages and serialization formats is safety. These tools are typically used in places that deal with untrusted inputs, and features and design choices can go a long way in either exposing or shielding developers from potential safety bugs. My read of Cap’n’Proto didn’t make it sound that safety was the highest priority. At least not above perfo…

I agree entirely, and this is one of my single greatest frustrations with the majority of the current popular IDLs/schema languages. ASN.1 is hilariously bad in a lot of ways, but one thing it gets absolutely right is strong typing and being able to express constraints (ranges, values dependent on other values). That combined with a canonicalized encoding form (DER) goes a long way in making various error states unre…

Except that ASN.1 is egregiously terrible at being able to be checked for wonky values due to complex parsing.

Exactly how many vulnerabilities have been exploited in LDAP, SNMP, etc. because ASN.1 is so terrible?

Re: A Critique of the Cap'n Proto Schema Language (2019)

#32

Post by formerly_proven » Sun, Aug 20, 2023, 9:35 AM UTC

signa11 wrote: post #15

jiggawatts wrote: post #10

Earlier quoted context omitted.

Safety means: garbage in, error out.

right, and it should not be left to the serialization layer for that.

Disregard for safety and security in serialization is one of the most common, if not the most common, cause for security vulnerabilities.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#33

Post by nly » Sun, Aug 20, 2023, 10:44 AM UTC

woodruffw wrote: post #14

ardel95 wrote: post #5

I think one of the most desirable and under-appreciated goals of schema languages and serialization formats is safety. These tools are typically used in places that deal with untrusted inputs, and features and design choices can go a long way in either exposing or shielding developers from potential safety bugs. My read of Cap’n’Proto didn’t make it sound that safety was the highest priority. At least not above perfo…

I agree entirely, and this is one of my single greatest frustrations with the majority of the current popular IDLs/schema languages. ASN.1 is hilariously bad in a lot of ways, but one thing it gets absolutely right is strong typing and being able to express constraints (ranges, values dependent on other values). That combined with a canonicalized encoding form (DER) goes a long way in making various error states unre…

I had to use it at work in a C++ environment and ended up settling on patching a copy of https://github.com/vlm/asn1c

Can't say I'd feel confident putting any of this stuff in a public service. Too complex and prone to bugs.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#34

Post by nly » Sun, Aug 20, 2023, 10:45 AM UTC

nullc wrote: post #16

ardel95 wrote: post #5

I think one of the most desirable and under-appreciated goals of schema languages and serialization formats is safety. These tools are typically used in places that deal with untrusted inputs, and features and design choices can go a long way in either exposing or shielding developers from potential safety bugs. My read of Cap’n’Proto didn’t make it sound that safety was the highest priority. At least not above perfo…

> and serialization formats is safety. These tools are typically used in places that deal with untrusted inputs, and features and design choices can go a long way in either exposing or shielding developers from potential safety bugs My potentially incorrect understanding is that Cap'n Proto's zero copy nature means the serialization format IS the in-memory representation, which means that if you build a Cap'n Proto o…

It's only zero copy to parse/read. The builders allocate all over the place.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#35

Post by woodruffw » Sun, Aug 20, 2023, 10:55 AM UTC

bsder wrote: post #31

woodruffw wrote: post #14

Earlier quoted context omitted.

I agree entirely, and this is one of my single greatest frustrations with the majority of the current popular IDLs/schema languages. ASN.1 is hilariously bad in a lot of ways, but one thing it gets absolutely right is strong typing and being able to express constraints (ranges, values dependent on other values). That combined with a canonicalized encoding form (DER) goes a long way in making various error states unre…

Except that ASN.1 is egregiously terrible at being able to be checked for wonky values due to complex parsing. Exactly how many vulnerabilities have been exploited in LDAP, SNMP, etc. because ASN.1 is so terrible?

ASN.1 isn’t an encoding; DER is.

The problem with LDAP, etc. is that they all permit BER, which is a looser superset of DER. It includes (among other things) the ability to represent indefinite-length fields, which are the single biggest source of exploitable bugs in a typical application of ASN.1. Without that, the exploitable surface of DER is much smaller (and especially when implemented in a memory-safe language).

Re: A Critique of the Cap'n Proto Schema Language (2019)

#36

Post by woodruffw » Sun, Aug 20, 2023, 10:59 AM UTC

debugnik wrote: post #27

woodruffw wrote: post #14

Earlier quoted context omitted.

I agree entirely, and this is one of my single greatest frustrations with the majority of the current popular IDLs/schema languages. ASN.1 is hilariously bad in a lot of ways, but one thing it gets absolutely right is strong typing and being able to express constraints (ranges, values dependent on other values). That combined with a canonicalized encoding form (DER) goes a long way in making various error states unre…

Is there a reasonable subset of ASN.1 that could get traction nowadays if specified separately?

There’s a wide set of best practices (use only DER for encoding, avoid legacy string types, etc.) that are widely applied in cryptographic applications, although I don’t know if anybody has written them down explicitly.

More generally: this wasn’t intended to be an endorsement of ASN.1 per se! It was only to say that it got some things right, things that Cap’n Proto and Protobuf appear to have eschewed. I’m not sure it is the right IDL for modern purposes, but I think it’s a useful piece of reference material.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#37

Post by kentonv » Sun, Aug 20, 2023, 11:22 AM UTC

ardel95 wrote: post #5

I think one of the most desirable and under-appreciated goals of schema languages and serialization formats is safety. These tools are typically used in places that deal with untrusted inputs, and features and design choices can go a long way in either exposing or shielding developers from potential safety bugs. My read of Cap’n’Proto didn’t make it sound that safety was the highest priority. At least not above perfo…

With all due respect, you read completely wrong.

* The very first use case for which Cap'n Proto was designed was to be the protocol that Sandstorm.io used to talk between sandbox and supervisor -- an explicitly adversarial security scenario.

* The documentation explicitly calls out how implementations should manage resource exhaustion problems like deep recursion depth (stack overflow risk), were many serialization formats leave these things as the app's problem.

* The implementation has been fuzz-tested multiple ways, including as part of Google's oss-fuzz.

* When there are security bugs, I issue advisories like this:

https://github.com/capnproto/capnproto/tree/v2/security-advi...

* The primary aim of the entire project is to be a Capability-Based Security RPC protocol. That's what "Cap" in the name comes from. The zero-copy serialization is actually a bonus feature.

(I'm the author of Cap'n Proto.)

Re: A Critique of the Cap'n Proto Schema Language (2019)

#38

Post by kentonv » Sun, Aug 20, 2023, 11:26 AM UTC

nly wrote: post #34

nullc wrote: post #16

Earlier quoted context omitted.

> and serialization formats is safety. These tools are typically used in places that deal with untrusted inputs, and features and design choices can go a long way in either exposing or shielding developers from potential safety bugs My potentially incorrect understanding is that Cap'n Proto's zero copy nature means the serialization format IS the in-memory representation, which means that if you build a Cap'n Proto o…

It's only zero copy to parse/read. The builders allocate all over the place.

A MessageBuilder allocates a single large buffer, writes into it, and only allocates further if that buffer is exhausted. If you use a preallocated buffer you can avoid allocation entirely. Very different from Protobuf which allocates strings, arrays, and sub-messages all as separate heap objects.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#39

Post by jnwatson » Sun, Aug 20, 2023, 11:37 AM UTC

bsder wrote: post #31

woodruffw wrote: post #14

Earlier quoted context omitted.

I agree entirely, and this is one of my single greatest frustrations with the majority of the current popular IDLs/schema languages. ASN.1 is hilariously bad in a lot of ways, but one thing it gets absolutely right is strong typing and being able to express constraints (ranges, values dependent on other values). That combined with a canonicalized encoding form (DER) goes a long way in making various error states unre…

Except that ASN.1 is egregiously terrible at being able to be checked for wonky values due to complex parsing. Exactly how many vulnerabilities have been exploited in LDAP, SNMP, etc. because ASN.1 is so terrible?

I've written an ASN.1 parser. The problem isn't the specification (though it is definitely a kitchen sink spec). The problem is the majority of ASN.1 code was written before the year 2000.

ASN.1 started in 1984. That means there are decades of shitty implementations, written well before adversarial input was considered a factor.

Re: A Critique of the Cap'n Proto Schema Language (2019)

#40

Post by jiggawatts » Sun, Aug 20, 2023, 11:42 AM UTC

signa11 wrote: post #29

jiggawatts wrote: post #26

Earlier quoted context omitted.

Security is a concern for every layer. It's not magic pixie dust that' can be sprinkled on top of software to renders it secure! A while ago I read a great article about how the Adobe PDF serialization format is nearly impossible to secure because it allows inherently unsafe constructs. For example, it allows cross-references that are basically just arbitrary unaligned pointers. It uses many different alignment and p…

> Serialization and parsing are security minefields and it is dangerously naive to just hand-wave that away. well, i am not hand-waving them away, i am not sure what can the serialization framework possibly _do_ to make things secure during the serialization ? when execution of user-supplied code is allowed (in the examples that you have outlined above), surely, the layer _executing_ the code cannot really do anythin…

> I am not sure what can the serialization framework possibly _do_ to make things secure during the serialization

Loads of things!

A strict specification that can only be interpreted one way goes very far. E.g.: a machine-readable BNF grammar file or something similar with no ambiguities.

A conformance test suite covering corner-cases is surprisingly effective, even with a supposedly perfect spec.

"Be strict with what you generate and lax with what you accept" has been demonstrated over and over again to be a disaster over the long-term in an ecosystem of many groups. Be strict always with what is accepted, not just generated!

Speaking of being strict: schema validation is essential. Strong typing for scalars helps a lot.

The actual implementations of the spec can obviously have a wide range of security features. Never allowing arbitrary type instantiation is critical, yet is a mistake that keeps reoccurring much like SQL injection.

Etc, etc...