Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The State of OpenSSL for pyca/cryptography

cryptography.io

Post reply on HN
Print view View original

21–30 of 65 posts

Re: The State of OpenSSL for pyca/cryptography

#21

Post by Xraider72 »

formerly_proven wrote: post #2

> Finally, taking an OpenSSL public API and attempting to trace the implementation to see how it is implemented has become an exercise in self-flagellation. Being able to read the source to understand how something works is important both as part of self-improvement in software engineering, but also because as sophisticated consumers there are inevitably things about how an implementation works that aren’t documented…

The whole 3.0 rewrite is a massive regression in all ways possible - they deprecated the old engines and replaced them with providers, and they are not that much easier to work with as a developer (I hope that providers are at least easier for the maintainers to handle) and the library is a lot more runtime dynamic (for some reason). This has resulted in mutex explosion and massive performance regressions in every facet. haproxy has an amusing article on the topic.

https://www.haproxy.com/blog/state-of-ssl-stacks

People who need cryptography but on the openssl API should be using aws-lc and seek a TLS stack elsewhere.

Re: The State of OpenSSL for pyca/cryptography

#22

Post by Xraider72 »

some_furry wrote: post #9
woodruffw wrote: post #3

I think this part is really worth engaging with: > Later, moving public key parsing to our own Rust code made end-to-end X.509 path validation 60% faster — just improving key loading led to a 60% end-to-end improvement, that’s how extreme the overhead of key parsing in OpenSSL was. > The fact that we are able to achieve better performance doing our own parsing makes clear that doing better is practical. And indeed, o…

Now I wonder how much performance is being left on the table elsewhere in the OpenSSL codebase...

haproxy has an article on the subject

https://www.haproxy.com/blog/state-of-ssl-stacks

TLDR - on the TLS parts, quite a lot, up to 2x slower on certain paths. Amusingly, openssl 1.1 was much faster.

libcrypto tends to be quite solid though, though over the years, other libraries have collected weird SIMD optimizations that enable them to beat openssl by healthy margins.

Re: The State of OpenSSL for pyca/cryptography

#24

Post by PunchyHamster »

tialaramex wrote: post #15
dfajgljsldkjag wrote: post #7

It is honestly surprising that OpenSSL has been the standard for so long given how difficult it is to work with. I think moving the backend to Rust is probably the right move for long term stability.

A couple of things probably made this more likely for OpenSSL than for other libraries, though I think this phenomenon (sticking with a famous library which just isn't very good) is overall just much more common than most people appreciate 1. OpenSSL is cryptography. We did explicitly tell people not to roll their own. So the first instinct of a programmer who finds X annoying ("Let's just write my own X") is ruled o…

but it isn't "rolling your own" but changing the lib you use.

> The C++ programmer who needs to do an HTTPS POST but mostly is doing 3D graphics could spend a month learning about the Web PKI, AES, the X.500 directory system and the Distinguished Encoding, or they could just call OpenSSL and not care.

they gonna call libcurl, not openssl directly. Tho they might use it still for parsing certs but that's easier to replace

Re: The State of OpenSSL for pyca/cryptography

#25

Post by nine_k »

jmspring wrote: post #11
tialaramex wrote: post #10

Earlier quoted context omitted.

It is extremely common that a correct implementation also has excellent performance. Also, even if somebody else can go faster by not being correct, what use is the wrong answer? https://nitter.net/magdraws/status/1551612747569299458

I’d say correct common path. OpenSSL due to hand waving deals with a lot of edge cases the correct path doesn’t handle. Even libraries like libnss suffers from this.

Are these edge cases correct to the spec, or not?

Re: The State of OpenSSL for pyca/cryptography

#26

Post by dochtman »

tialaramex wrote: post #15
dfajgljsldkjag wrote: post #7

It is honestly surprising that OpenSSL has been the standard for so long given how difficult it is to work with. I think moving the backend to Rust is probably the right move for long term stability.

A couple of things probably made this more likely for OpenSSL than for other libraries, though I think this phenomenon (sticking with a famous library which just isn't very good) is overall just much more common than most people appreciate 1. OpenSSL is cryptography. We did explicitly tell people not to roll their own. So the first instinct of a programmer who finds X annoying ("Let's just write my own X") is ruled o…

There’s also a compounding effect: I’ve heard from a hardware vendor that they spend a lot of time optimizing OpenSSL to get the most out of their silicon, so for their customers they suggest using OpenSSL to get the most out of the hardware.

Re: The State of OpenSSL for pyca/cryptography

#27

Post by ivanr »

I wrote about OpenSSL's performance regressions in the December issue of Feisty Duck's cryptography newsletter [1]. In addition to Alex's and Paul's talk on Python cryptography, at the recent OpenSSL conference there have been several other talks worth watching:

- William Bellingrath, from Juniper Networks, benchmarked versions from 1.1.1 to 3.4.x https://www.youtube.com/watch?v=b01y5FDx-ao

- Tomáš Mráz wrote about how to get better performance, which, in turn, explains why it's bad by default: https://www.youtube.com/watch?v=Cv-43gJJFIs

- Martin Schmatz from IBM presented about their _very detailed_ study of post-quantum cipher suite performance https://www.youtube.com/watch?v=69gUVhOEaVM

Note: be careful with these recorded talks as they have a piercing violin sound at the beginning that's much louder than the rest. I've had to resort to muting the first couple of seconds of every talk.

[1] https://www.feistyduck.com/newsletter/issue_132_openssl_perf...

Re: The State of OpenSSL for pyca/cryptography

#28

Post by timschmidt »

PunchyHamster wrote: post #23
captain_coffee wrote: post #19

I had no idea that OpenSSL is in such a bad state.

I was surprised it is still in such bad state even after "rewrite" for 3.0.

I am not surprised at all, because instead of throwing their support behind the LibreSSL folks who audited the OpenSSL codebase after Heartbleed and found deep design and implementation issues, Linux Foundation and member orgs including most of Silicon Valley decided that OpenSSL just needed more funding.

Felt like good money after bad on day 1.

Re: The State of OpenSSL for pyca/cryptography

#29

Post by pseudohadamard »

nine_k wrote: post #25
jmspring wrote: post #11

Earlier quoted context omitted.

I’d say correct common path. OpenSSL due to hand waving deals with a lot of edge cases the correct path doesn’t handle. Even libraries like libnss suffers from this.

Are these edge cases correct to the spec, or not?

Yes.

The spec is often such a confused mess that even the people who wrote it are surprised by what it requires. One example was when someone on the PKIX list spent some time explaining to X.509 standards people what it was that their own standard required, which they had been unaware of until then.

Post reply on HN