Live data from Hacker News

The database servers powering Let's Encrypt

letsencrypt.org

111–120 of 236 posts

Re: The database servers powering Let's Encrypt

#111
@jaas I wonder how many servers you have and in particular how a failure of a system would play out in your situation.

The latter is always interesting to me, building things is easy. Building rock solid reliable things is seriously hard work and often difficult.

Would make a great blog post....

Re: The database servers powering Let's Encrypt

#112
post #88
post #68

+10% for the proper use of decimated

You mean, that one? https://en.wikipedia.org/wiki/Decimation_(Roman_army) Then it would be 90 ms -> 81 ms, not 90 ms -> 9 ms. The way I see it, at least. With proper decimation, 90% of what was there remains. ("removal of a tenth", as wikipedia puts it).

Oh man, you’re totally right. I ... uh, blame somebody else.

Re: The database servers powering Let's Encrypt

#114
post #25

I was, long ago, an old-school Unix sysadmin. While I was technically aware of how powerful smallish servers have become, this article really crystallized that for me. 64 cores and 24 NVME drives in a 2U spot on a rack is just insane compared to what we used to have to do to get a beefy database server. And it's not some exotic thing, just a popular mainstream Dell SKU. If you price it out on Dell's site, you get a r…

STH recently did a video about this dense mass of storage & compute they call a server. iirc they specced it with their discounts to end up at a much more reasonable ~$90k.

https://youtu.be/9PLvt3WATMU

Re: The database servers powering Let's Encrypt

#115
>Intel® SSD DC P4610

Interesting they decide to put in PCIE 3.0 NVMe SSD instead of PCI-E 4?

Imagine having 24x Intel Optane [1]. PCI-E 5 is actually just around the corner. I would imagine next time Let's Encrypt could upgrade again and continue to use a Single DB Machine to Serve.

[1] https://www.servethehome.com/new-intel-optane-p5800x-100-dwp...

Re: The database servers powering Let's Encrypt

#116
post #5

What are they storing on this server that requires 150Tb of storage and millions of IOPS?

My thoughts exactly ... They are creating and storing text. Only thing I can think is that they don't actually need the storage, but just want the lowest possible latency by having a large number of drives.

Well if Latency were an issue or something they like to lower they should have chosen Optane.

Re: The database servers powering Let's Encrypt

#117
post #93
post #25

I was, long ago, an old-school Unix sysadmin. While I was technically aware of how powerful smallish servers have become, this article really crystallized that for me. 64 cores and 24 NVME drives in a 2U spot on a rack is just insane compared to what we used to have to do to get a beefy database server. And it's not some exotic thing, just a popular mainstream Dell SKU. If you price it out on Dell's site, you get a r…

Totally. 2 TB of RAM! In one box! I think the first servers I had in production had 8 MB RAM. No more, certainly. Soon we'll be at 1000x that. My dad's first "server" was 3 orders of magnitude smaller, with 8 KB of RAM (hand-wound wire core memory). In that time, the US population hasn't even doubled.

https://yourdatafitsinram.net/

The listed servers are from few years ago so I guess there might be some bigger ones available these days.

Re: The database servers powering Let's Encrypt

#118

Earlier quoted context omitted.

Based on their stated 225M sites and a renewal period of 90 days, they're probably averaging around 40 certificates per second. That's only an order of magnitude higher than bitcoin; I wouldn't call it an indication of an ability to scale to a particularly large amount of traffic.

Yes. They are not doing a very heavy computational workload. Typical heavy-duty servers these days can do 100k's or millions of TPS. 40 TPS is a really, really, really light load. Further, I was looking at those new server specs. There's an error I think? The server config on the Dell site shows 2x 8 GB DRIMMs, for 16 GB RAM per sever, whereas the article says 2 TB! With only 16GB of RAM, but 153.6 TB of NVMe storage…

Don’t these certificates have long RSA keys which are more expensive computationally? Though I guess that doesn’t have to happen on the database server.

Re: The database servers powering Let's Encrypt

#119
post #93
post #25

I was, long ago, an old-school Unix sysadmin. While I was technically aware of how powerful smallish servers have become, this article really crystallized that for me. 64 cores and 24 NVME drives in a 2U spot on a rack is just insane compared to what we used to have to do to get a beefy database server. And it's not some exotic thing, just a popular mainstream Dell SKU. If you price it out on Dell's site, you get a r…

Totally. 2 TB of RAM! In one box! I think the first servers I had in production had 8 MB RAM. No more, certainly. Soon we'll be at 1000x that. My dad's first "server" was 3 orders of magnitude smaller, with 8 KB of RAM (hand-wound wire core memory). In that time, the US population hasn't even doubled.

My phone in my pocket has 1,000 times 8MB of ram. A million of your first serves in one box. In about 30 years?

Re: The database servers powering Let's Encrypt

#120
post #113

I don't understand why they are trying so hard to avoid sharding. It seems to me that this is a perfect example of an "embarassingly parallel" problem for which sharding would be borderline trivial. What am I missing?

Well, their current solution is still a lot simpler, I suppose. With multiple servers there's a lot of extra administrative stuff you need to deal with.
Post reply on HN