Live data from Hacker News

The database servers powering Let's Encrypt

letsencrypt.org

161–170 of 236 posts

Re: The database servers powering Let's Encrypt

#161
post #139

Earlier quoted context omitted.

We made so much progress in that field over the last 40 years... I sometimes get lost trying to imagine what will be the computing performance available in 100, 1000, 10000 years from now...

Single threaded performance isn't keeping pace unfortunately.

That just means multithreaded will become more and more important. Languages like rust may gain some usage as we start to require safe multithreading.

Re: The database servers powering Let's Encrypt

#162

Earlier quoted context omitted.

Based on their stated 225M sites and a renewal period of 90 days, they're probably averaging around 40 certificates per second. That's only an order of magnitude higher than bitcoin; I wouldn't call it an indication of an ability to scale to a particularly large amount of traffic.

They might average that, but we all know averages only work on paper. For example, AWS has tutorials that provide instructions on how to setup TLS in a LAMP stack running on Linux 2 EC2s. As part of the Let's Encrypt setup, they provide a crontab entry that runs twice a day with a copy option to paste. How many EC2s all hit the Let's Encrypt server at that exact time? Since EC2s default to UTC time, that means that s…

Its worth noting that running certbot / LE twice a day doesn't actually hit the LE server twice a day. It just checks the certificate dates locally and if they have been renewed in the last month it does nothing.

I guess you still have a peak time of 00:00 UTC every day though unless people are using servers set to their local time.

Re: The database servers powering Let's Encrypt

#163

Earlier quoted context omitted.

We made so much progress in that field over the last 40 years... I sometimes get lost trying to imagine what will be the computing performance available in 100, 1000, 10000 years from now...

Sticks and rocks most likely.

Yep, quantum entanglement materials thru and thru

Re: The database servers powering Let's Encrypt

#164

Earlier quoted context omitted.

Based on their stated 225M sites and a renewal period of 90 days, they're probably averaging around 40 certificates per second. That's only an order of magnitude higher than bitcoin; I wouldn't call it an indication of an ability to scale to a particularly large amount of traffic.

Bitcoin is ECDSA verification, letsencrypt is generating RSA signatures, the two aren't even remotely comparable.

Bitcoin is actually not limited by compute power at all. Its an artificial cap on transaction rate to prevent the blockchain from expanding too large and preventing normal users from hosting the whole thing.

You can see the blockchain size was growing exponentially but then switches to linear as we hit the transaction cap and it now sits at about 350GB

Re: The database servers powering Let's Encrypt

#165
post #147

Earlier quoted context omitted.

after 15 seconds of googling, the P4610 U.2 format 6.4TB seems something more like a single unit street price of $2400 from non-Dell vendors. I'm mildly surprised it's that low considering that the U.2 format stuff, for serious servers, will always command a premium price. Probably in the range of $2100 to $2200 per unit from a x86-64 component distributor in moderate quantities.

Just noticed this in the Dell cart... "42% off list price: use code SERVER42" Doesn't make the price reasonable exactly, but it's kind of funny.

You should try server43

Re: The database servers powering Let's Encrypt

#166
post #25

I was, long ago, an old-school Unix sysadmin. While I was technically aware of how powerful smallish servers have become, this article really crystallized that for me. 64 cores and 24 NVME drives in a 2U spot on a rack is just insane compared to what we used to have to do to get a beefy database server. And it's not some exotic thing, just a popular mainstream Dell SKU. If you price it out on Dell's site, you get a r…

For a comparison, you could get a 64 processor, 256Gb ram Sun Starfire around 20 years ago[1]. Wikipedia claims these cost well over a million dollars ($1.5 million in 2021 dollars). This machine was enormous (bigger than a rack), would have had more non-uniform memory access to deal with, and the processors were clocked at something like 250-650MHz. [1] https://en.wikipedia.org/wiki/Sun_Enterprise

That's a pretty good comparison. It weighed 2000lbs, and was 38 inches wide, or basically 2 full racks, which I guess you could call an 84U server. It was also 49 inches deep, versus a standard rack which is 36 inches deep.

Re: The database servers powering Let's Encrypt

#167
> We have a number of replicas of the database active at any given time, and we direct some read operations to replica database servers to reduce load on the primary.

No shared storage, no storage efficiency, and RF2 replication with mySQL on top of that... Ouch.

And it's completely unclear why NVMe was necessary in the first place. Are they using more than 5% of its performance? Instead they talk about PCI lanes and whatnot.

Re: The database servers powering Let's Encrypt

#169
post #25

I was, long ago, an old-school Unix sysadmin. While I was technically aware of how powerful smallish servers have become, this article really crystallized that for me. 64 cores and 24 NVME drives in a 2U spot on a rack is just insane compared to what we used to have to do to get a beefy database server. And it's not some exotic thing, just a popular mainstream Dell SKU. If you price it out on Dell's site, you get a r…

We made so much progress in that field over the last 40 years... I sometimes get lost trying to imagine what will be the computing performance available in 100, 1000, 10000 years from now...

And I'm afraid it will still obey this trend.

https://danluu.com/input-lag/

Re: The database servers powering Let's Encrypt

#170

Earlier quoted context omitted.

Based on their stated 225M sites and a renewal period of 90 days, they're probably averaging around 40 certificates per second. That's only an order of magnitude higher than bitcoin; I wouldn't call it an indication of an ability to scale to a particularly large amount of traffic.

Yes. They are not doing a very heavy computational workload. Typical heavy-duty servers these days can do 100k's or millions of TPS. 40 TPS is a really, really, really light load. Further, I was looking at those new server specs. There's an error I think? The server config on the Dell site shows 2x 8 GB DRIMMs, for 16 GB RAM per sever, whereas the article says 2 TB! With only 16GB of RAM, but 153.6 TB of NVMe storage…

Why are you assuming that their workload includes just one query per emitted certificate?

The reality is that they are storing information during challenges, implementing rate limiting per-account, supporting OCSP validation and a few other things.

You can investigate further if you really want to see the queries that they make against the database since their software (Boulder) is open source [1]. Most queries are in the files in the "sa" (storage authority) folder.

[1] https://github.com/letsencrypt/boulder/

Post reply on HN