The database servers powering Let's Encrypt • Hacker News Board

Re: The database servers powering Let's Encrypt

#81

Post by fipar » Thu, Jan 21, 2021, 9:51 PM UTC

jeffbee wrote: post #32

kirugan wrote: post #6

Earlier quoted context omitted.

MyISAM days are gone, no one will seriously consider it as suitable engine in MySQL.

It is quite likely unless you've meticulously avoided it that MySQL is using ISAM on-disk temp tables in the service of your queries.

Actually, that shouldn't be the case since 5.7: https://dev.mysql.com/doc/refman/5.7/en/server-system-variab... (and related: https://dev.mysql.com/doc/refman/5.7/en/server-system-variab...)

And on 8 MyISAM is mostly gone, not even the 'mysql' schema uses it.

Edit: Originally linked only to default_tmp_storage_engine).

Re: The database servers powering Let's Encrypt

#82

Post by siwyd » Thu, Jan 21, 2021, 9:53 PM UTC

Conclusionist wrote: post #65

stefan_ wrote: post #64

Earlier quoted context omitted.

What exactly needs to be stored once the certificate is created and published in the hash tree? It seems like the kind of data that possibly needn't be stored at all or onto something like Glacier for archival.

Going to guess it's for OCSP responses.

FYI, the intermediate CA's signed by their new Root X2 certificate won't have OCSP URLs anymore.

Source: https://letsencrypt.org/2020/09/17/new-root-and-intermediate...

Re: The database servers powering Let's Encrypt

#83

Post by theandrewbailey » Thu, Jan 21, 2021, 9:56 PM UTC

baby wrote: post #31

So scaling up instead of scaling out. I’m not sure if it’s a viable strategy long term, at the same time we probably don’t want a single CA to handle too many certificates?

I'm not aware of any other CA giving out free certificates to anyone. I know that some other providers/hosts will do free certificates, but only to their users (last time I checked).

Re: The database servers powering Let's Encrypt

#84

Post by birdman3131 » Thu, Jan 21, 2021, 9:57 PM UTC

jabberwcky wrote: post #67

cbg0 wrote: post #10

Unless I misunderstood something, it seems they have a single primary that handles read+write and multiple read replicas for it. It shouldn't be too difficult given the current use of MariaDB to start using something like Galera to create a multi-master cluster and improve redundancy of the service, unless there are some non-obvious reasons why they wouldn't be doing this. I think I also see redundant PSUs, would be…

Multi-master hardly comes for free in terms of complexity or performance, you're at the mercy of latency. Either host the second master in the same building, in which case the redundancy is an illusion, or host it somewhere else in which case watch your write rate tank Asynchronous streaming to a truly redundant second site often makes more sense

How well would same city with fiber between work?

Re: The database servers powering Let's Encrypt

#85

Post by gpsar » Thu, Jan 21, 2021, 9:59 PM UTC

tanelpoder wrote: post #51

tanelpoder wrote: post #36

Earlier quoted context omitted.

Yes, with modern storage, throughput is a CPU problem. And CPU problem for OLTP databases is largely a memory access latency problem. For columnar analytics & complex calculations it's more about CPU itself. When doing 1 MB sized I/Os for scanning, my 16c/32t (AMD Ryzen Threadripper Pro WX) CPUs were just about 10% busy. So, with a 64 core single socket ThreadRipper workstation (or 128-core dual socket EPYC server),…

As I mentioned memory access latency - I just posted my old article series about measuring RAM access performance (using different database workloads) to HN and looks like it even made it to the front page (nice): https://news.ycombinator.com/item?id=25863093

If the problem involves independent traversals, interleaving with coroutines is a practical way to hide latency https://dl.acm.org/doi/10.1145/3329785.3329917 https://www.linkedin.com/pulse/dont-stall-multitask-georgios...

Re: The database servers powering Let's Encrypt

#86

Post by [deleted] » Thu, Jan 21, 2021, 10:00 PM UTC

dingaling wrote: post #80

barkingcat wrote: post #9

Earlier quoted context omitted.

For a service like letsencrypt, the independence factor is also a major reason for self hosting. I can forsee letsencrypt in the future going to building their own cloud (on their own physical infrastructure), but speaking as a letsencrypt user of their free certificate program, I would lose respect and interest in their service if they went with an AWS or GCP or Azure approach. The independence from other major play…

Funny you mention AWS as they're one of the corporate sponsors of LE. So long as they don't have a viable independent revenue stream they're arguably less independent than commercial CAs.

[deleted]

Re: The database servers powering Let's Encrypt

#87

Post by theandrewbailey » Thu, Jan 21, 2021, 10:00 PM UTC

tyingq wrote: post #25

I was, long ago, an old-school Unix sysadmin. While I was technically aware of how powerful smallish servers have become, this article really crystallized that for me. 64 cores and 24 NVME drives in a 2U spot on a rack is just insane compared to what we used to have to do to get a beefy database server. And it's not some exotic thing, just a popular mainstream Dell SKU. If you price it out on Dell's site, you get a r…

Dell-controlled VMWare is listed under "major sponsors and funders". I wonder if Let's Encrypt got a discount on that server. Good for them!

Re: The database servers powering Let's Encrypt

#88

Post by MayeulC » Thu, Jan 21, 2021, 10:01 PM UTC

tclancy wrote: post #68

+10% for the proper use of decimated

You mean, that one? https://en.wikipedia.org/wiki/Decimation_(Roman_army)

Then it would be 90 ms -> 81 ms, not 90 ms -> 9 ms. The way I see it, at least. With proper decimation, 90% of what was there remains. ("removal of a tenth", as wikipedia puts it).

Re: The database servers powering Let's Encrypt

#89

Post by vmception » Thu, Jan 21, 2021, 10:05 PM UTC

foota wrote: post #73

vmception wrote: post #48

Earlier quoted context omitted.

I have a motherboard from 2012 and I just put 2x 8TB NVMe SSDs on it, on a PCIe 2.0 x16 slot Works great. The PCIe card itself has 2 more slots for SSDs The GPU is on the 2.0 x8 slot because they don't really transfer that much data over the lanes. I honestly didn't realize PCIe was up to 4.0 now, and I am pushing up against the limits of PCIe 2.0 but it still works! And I’m “only” at the limits, and its only a limit…

What drives did you get? I think you need PCI 4 to stress most SSDs these days?

I have a hunch that the pcie card itself is most important as it is doing bifurcation.

So each drive acts like it has its own slower (but fast enough) pcie slot, and then the raid0 combines the bits back to double the performance.

Could be wrong but I get 2,900 megabytes per second transfers from RAM to disk and back.

And this is PCIe 2.0 x16

so maybe if you want , 3,000, 4,000 or 6,500 megabytes per second then I have nothing to brag about. I’m pretty amazed though and will be content for all my use cases.

Re: The database servers powering Let's Encrypt

#90

Post by cipherboy » Thu, Jan 21, 2021, 10:05 PM UTC

siwyd wrote: post #82

Conclusionist wrote: post #65

Earlier quoted context omitted.

Going to guess it's for OCSP responses.

FYI, the intermediate CA's signed by their new Root X2 certificate won't have OCSP URLs anymore. Source: https://letsencrypt.org/2020/09/17/new-root-and-intermediate...

AFAIK, nobody has suggested removal of OCSP from end-entity certificates. This article you linked (and the comment you wrote) is purely about removal from intermediate CA certificates.

The majority of OCSP traffic will probably be for end-entity certificates; most OCSP validation (in browsers and cryptographic libraries) is end-entity validation, not leaf-and-chain.

Removal of intermediate CA's OCSP is probably not really relevant to their overall OCSP performance numbers (and if it was, it was likely cached already).