Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The State of OpenSSL for pyca/cryptography

cryptography.io

Post reply on HN
Print view View original

11–20 of 65 posts

Re: The State of OpenSSL for pyca/cryptography

#11

Post by jmspring »

tialaramex wrote: post #10
woodruffw wrote: post #3

I think this part is really worth engaging with: > Later, moving public key parsing to our own Rust code made end-to-end X.509 path validation 60% faster — just improving key loading led to a 60% end-to-end improvement, that’s how extreme the overhead of key parsing in OpenSSL was. > The fact that we are able to achieve better performance doing our own parsing makes clear that doing better is practical. And indeed, o…

It is extremely common that a correct implementation also has excellent performance. Also, even if somebody else can go faster by not being correct, what use is the wrong answer? https://nitter.net/magdraws/status/1551612747569299458

I’d say correct common path. OpenSSL due to hand waving deals with a lot of edge cases the correct path doesn’t handle. Even libraries like libnss suffers from this.

Re: The State of OpenSSL for pyca/cryptography

#12

Post by woodruffw »

tialaramex wrote: post #10
woodruffw wrote: post #3

I think this part is really worth engaging with: > Later, moving public key parsing to our own Rust code made end-to-end X.509 path validation 60% faster — just improving key loading led to a 60% end-to-end improvement, that’s how extreme the overhead of key parsing in OpenSSL was. > The fact that we are able to achieve better performance doing our own parsing makes clear that doing better is practical. And indeed, o…

It is extremely common that a correct implementation also has excellent performance. Also, even if somebody else can go faster by not being correct, what use is the wrong answer? https://nitter.net/magdraws/status/1551612747569299458

> It is extremely common that a correct implementation also has excellent performance.

I think that's true in general, but in the case of X.509 path validation it's not a given: the path construction algorithm is non-trivial, and requires quadratic searches (e.g. of name constraints against subjects/SANs). An incorrect implementation could be faster by just not doing those things, which is often fine (for example, nothing really explodes if an EE doesn't have a SAN[1]). I think one of the things that's interesting in the PyCA case is that it commits to doing a lot of cross-checking/policy work that is "extra" on paper but stills comes out on top of OpenSSL.

[1]: https://x509-limbo.com/testcases/webpki/#webpkisanno-san

Re: The State of OpenSSL for pyca/cryptography

#13

Post by ameliaquining »

dfajgljsldkjag wrote: post #7

It is honestly surprising that OpenSSL has been the standard for so long given how difficult it is to work with. I think moving the backend to Rust is probably the right move for long term stability.

Note that all cryptographic primitives are still going to be in C via an OpenSSL-like API for the next while; the current proposal is to migrate from OpenSSL to one of its forks. Various bits of backend logic that aren't cryptographic primitives (e.g., parsing) have been rewritten in Rust; additionally, https://github.com/ctz/graviola is mentioned near the end as a possible implementation of cryptographic primitives in a combination of Rust and assembly (without any C), but it's not especially mature yet.

Re: The State of OpenSSL for pyca/cryptography

#14

Post by Avamander »

some_furry wrote: post #9
woodruffw wrote: post #3

I think this part is really worth engaging with: > Later, moving public key parsing to our own Rust code made end-to-end X.509 path validation 60% faster — just improving key loading led to a 60% end-to-end improvement, that’s how extreme the overhead of key parsing in OpenSSL was. > The fact that we are able to achieve better performance doing our own parsing makes clear that doing better is practical. And indeed, o…

Now I wonder how much performance is being left on the table elsewhere in the OpenSSL codebase...

Given the massive regression with 3.x alone, you'll probably be happier if you don't know :/

Re: The State of OpenSSL for pyca/cryptography

#15

Post by tialaramex »

dfajgljsldkjag wrote: post #7

It is honestly surprising that OpenSSL has been the standard for so long given how difficult it is to work with. I think moving the backend to Rust is probably the right move for long term stability.

A couple of things probably made this more likely for OpenSSL than for other libraries, though I think this phenomenon (sticking with a famous library which just isn't very good) is overall just much more common than most people appreciate

1. OpenSSL is cryptography. We did explicitly tell people not to roll their own. So the first instinct of a programmer who finds X annoying ("Let's just write my own X") is ruled out by this as likely unwise or attracts backlash from their users, "What do you mean you rolled your own TLS implementation?"

2. Even the bits which aren't cryptography are niches likely entirely unrelated to the true interest of the author using OpenSSL. The C++ programmer who needs to do an HTTPS POST but mostly is doing 3D graphics could spend a month learning about the Web PKI, AES, the X.500 directory system and the Distinguished Encoding, or they could just call OpenSSL and not care.

Re: The State of OpenSSL for pyca/cryptography

#16

Post by RiverCrochet »

woodruffw wrote: post #3

I think this part is really worth engaging with: > Later, moving public key parsing to our own Rust code made end-to-end X.509 path validation 60% faster — just improving key loading led to a 60% end-to-end improvement, that’s how extreme the overhead of key parsing in OpenSSL was. > The fact that we are able to achieve better performance doing our own parsing makes clear that doing better is practical. And indeed, o…

Remember LibreSSL? That was borne of Heartbleed IIRC, and I remember presentation slides saying there was stuff in OpenSSL to support things like VAX, Amiga(?) and other ancient architectures. So I wonder if some of the things are there because of that.

Re: The State of OpenSSL for pyca/cryptography

#17

Post by tialaramex »

RiverCrochet wrote: post #16
woodruffw wrote: post #3

I think this part is really worth engaging with: > Later, moving public key parsing to our own Rust code made end-to-end X.509 path validation 60% faster — just improving key loading led to a 60% end-to-end improvement, that’s how extreme the overhead of key parsing in OpenSSL was. > The fact that we are able to achieve better performance doing our own parsing makes clear that doing better is practical. And indeed, o…

Remember LibreSSL? That was borne of Heartbleed IIRC, and I remember presentation slides saying there was stuff in OpenSSL to support things like VAX, Amiga(?) and other ancient architectures. So I wonder if some of the things are there because of that.

The Amigans really like their system. So they kept using them long after mainstream users didn't care. By now there probably aren't many left, but certainly when LibreSSL began there are still enough Amigans, actually using an Amiga to do stuff like browse web pages at least sometimes, that OpenSSL for Amiga kinda makes sense.

I mean, it still doesn't make sense, the Amigans should sort out their own thing, but if you're as into stamp collecting as OpenSSL is I can see why you'd be attracted to Amiga support.

Twenty years ago, there are Amigans with this weird "AmigaOne" PowerPC board that they've been told will some day hook to their legitimate 20th century Commodore A1200 Amiga. Obviously a few hundred megahertz of PowerPC is enough to attempt modern TLS 1.0 (TLS 1.1 won't be out for a while yet) and in this era although some web sites won't work without some fancy PC web browser many look fine on the various rather elderly options for Amigans and OpenSSL means that includes many login pages, banking, etc.

By ten years ago which is about peak LibreSSL, the Amigans are buying the (by their standards) cheaper AmigaOne 500, and the (even by their standards) expensive AmigaOne X5000. I'd guess there are maybe a thousand of them? So not loads, but that's an actual audience. The X5000 has decent perf by the standards of the day, although of course that's not actually available to an Amiga user, you've bought a dual-core 64-bit CPU but you can only use 32 bit addressing and one core because that's Amiga.

Re: The State of OpenSSL for pyca/cryptography

#18

Post by fanf2 »

RiverCrochet wrote: post #16
woodruffw wrote: post #3

I think this part is really worth engaging with: > Later, moving public key parsing to our own Rust code made end-to-end X.509 path validation 60% faster — just improving key loading led to a 60% end-to-end improvement, that’s how extreme the overhead of key parsing in OpenSSL was. > The fact that we are able to achieve better performance doing our own parsing makes clear that doing better is practical. And indeed, o…

Remember LibreSSL? That was borne of Heartbleed IIRC, and I remember presentation slides saying there was stuff in OpenSSL to support things like VAX, Amiga(?) and other ancient architectures. So I wonder if some of the things are there because of that.

Most of the performance regressions are due to lots of dynamic reconfigurability at runtime, which isn’t needed for portability to ancient systems. (Although OpenSSL is written in C it has a severe case of dynamic language envy, so it’s ironic that the pyca team want a less dynamic crypto library.)

Re: The State of OpenSSL for pyca/cryptography

#20

Post by toast0 »

dfajgljsldkjag wrote: post #7

It is honestly surprising that OpenSSL has been the standard for so long given how difficult it is to work with. I think moving the backend to Rust is probably the right move for long term stability.

Around the time of Heartbleed, pretty much nobody else wanted to do the work, and when they did, it was worse (GNU TLS).

The crypto primitives in OpenSSL tend to be pretty good, but the protocol stuff isn't great. x.509 is terrible, so something someone else wrote to deal with it is mighty tempting. TLS protocol isn't as bad, but seeing how many bytes are spent on length can drive someone crazy.

OpenSSL has historically been crap with respect to development compatability[1], but I think the terrible performance in the 3.x series pushed a lot of people over the edge. Do the protocol work, including x.509 in a memory safe language, manage locking yourself and call out to (a fork of) openssl for the crypto.

[1] Heartbleed would have been a lot worse if people weren't slowrolling upgrading to vulnerable versions because upgrading would be a pain

Post reply on HN