Earlier quoted context omitted.
You should try server43
server100 for the win
server1000 and we can retire early
The database servers powering Let's Encrypt
201–210 of 236 posts
Re: The database servers powering Let's Encrypt
#202Earlier quoted context omitted.
If you eschew RDS, the largest you can go up to seems to be a u-24tb1.metal. 448 vcpu, 24TiB of RAM, $70 an hour. ~$52k per month.
Oh, only $70 per hour. I think I can start that for an hour or so before my wallet is exhausted :P
I spun up a 32 vCore spot instance to crack my own WiFi password once. It only cost ~$2 and I felt so powerful.
Re: The database servers powering Let's Encrypt
#203I'm thankful for their OpenZFS tuning doc which they developed as part of this server migration: https://github.com/letsencrypt/openzfs-nvme-databases The one thing that I get hung up on when it comes to RAID and SSDs is the wear pattern vs. HDDs. Take for example this quote from the README.md: We use RAID-1+0, in order to achieve the best possible performance without being vulnerable to a single-drive failure. Failu…
The tuning is almost identical to what we have in production. A few comments:
* I would probably go with ashift=12 to get better compression ratio, or even as far as ashift=9 if the disks can sustainably maintain the same performance. Benchmark first, of course.
* We came to the same conclusion regarding AIO recently, but just today I did more benchmark, and it looks like ZFS shim does perform better than InnoDB shim. So I think it's still fine to enable innodb_use_native_aio
* We use zstd compression from ZFS 2.0. It's great, and we all deserve it after suffering through the PR dramas.
Re: The database servers powering Let's Encrypt
#204Earlier quoted context omitted.
Just so everyone here is aware re: pricing... If you buy through a VAR and/or Dell reps you don't pay the price on the website. What you actually pay is typically significantly lower. I don't think anyone actually buys servers like these by just ordering from the website. We (Let's Encrypt) certainly don't. These are expensive servers, crossing into six digits, but not $200k.
Literally the only reason to display a retail price on the website is so sales reps can say "see we are giving you a massive XX% discount!"
The same bullshit as US medical prices, it seems:
- Hospitals have an interest in massively inflated high sticker prices to make as much money as possible from those who pay in cash, e.g. foreigners, and to make the debts look better for collection agencies - basically, assuming the true cost of a procedure is 1000$, when the hospital bills 1000$ and that goes to collections who buy the debt for 10% of value the hospital gets 100$, and if the hospital bills 10k $ and the collections agency pays 10%, the hospital gets 1000$ or the actual cost
- Insurances have an interest in high sticker prices because they will negotiate with the hospital to pay true cost + some markup anyway - so with a higher sticker price, they can claim that their insurance saves the buyers a higher percentage
- Employers have an interest in high sticker prices because they can market themselves as employers who provide a better health insurance than the competition
The people losing out in this gamble are those who cannot afford insurance and have to declare medical bankruptcy.
And in the case of hardware or even some "contact sales for a quote" SaaS it's the same end result: the ones who lose out are small businesses (who can't achieve the sales amount to qualify for cheap-ish rates), and the big companies with dedicated account managers have a nice life.
Re: The database servers powering Let's Encrypt
#205I don't understand why they are trying so hard to avoid sharding. It seems to me that this is a perfect example of an "embarassingly parallel" problem for which sharding would be borderline trivial. What am I missing?
Sharding is like amputating a limb ! It's works really well if the limb is infected with say a flesh-eating bacteria ! But you keep sharding as a last last resort ! It's hard going back
Re: The database servers powering Let's Encrypt
#206Earlier quoted context omitted.
Just so everyone here is aware re: pricing... If you buy through a VAR and/or Dell reps you don't pay the price on the website. What you actually pay is typically significantly lower. I don't think anyone actually buys servers like these by just ordering from the website. We (Let's Encrypt) certainly don't. These are expensive servers, crossing into six digits, but not $200k.
Literally the only reason to display a retail price on the website is so sales reps can say "see we are giving you a massive XX% discount!"
But don't they lose sales from people who never bother to call a sales rep because they think it's out of their price range?
Re: The database servers powering Let's Encrypt
#207Lets for the sake of argument assume that Lets Encrypt is a malicious actor. Can they easily compromise the security of the websites using their certificates?
Re: The database servers powering Let's Encrypt
#208I'm thankful for their OpenZFS tuning doc which they developed as part of this server migration: https://github.com/letsencrypt/openzfs-nvme-databases The one thing that I get hung up on when it comes to RAID and SSDs is the wear pattern vs. HDDs. Take for example this quote from the README.md: We use RAID-1+0, in order to achieve the best possible performance without being vulnerable to a single-drive failure. Failu…
An early mitigation strategy has been to use different brands of ssd so that failure rates get more spread out. For raid 6 we have started out with max 2 drives of a single brand.
The result for us was 2 drives that failed within the same month, of the same brand, and from there it seems to be single failures rather than clusters.
Re: The database servers powering Let's Encrypt
#209Lets for the sake of argument assume that Lets Encrypt is a malicious actor. Can they easily compromise the security of the websites using their certificates?
Yes. Not just websites using their own certificates though. As a certificate authority they can create certificates for arbitrary domains. There are however a few countermeasures against illegitimate certificates such as certificate pinning and certificate transparency.
Re: The database servers powering Let's Encrypt
#210Earlier quoted context omitted.
And I'm afraid it will still obey this trend. https://danluu.com/input-lag/
That doesen't seem like some fundamental restriction, but rather a compromise we're always willing to make. 30ms is not noticeable, so we don't try to lower it and sacrifice something else. Even the super slow ones like on kindle, are _choices_ that have been made in favor of something else. a second to turn a page on a book isn't unbearable
30ms latency is acceptable. But that's the best we ever had.
High-end Android phones have 120+ ms latecy. That's easily noticeable and actually annoying (at least to me).
My personal pet peeve is the latency of input when i'm starting new applications in KDE. E.g. I'll start a new terminal with Super+Enter, followed by a Super+Right Arrow in order to tile it to the right. But the latency is big enough, that often it's not the terminal that ends up tiled, but the application that had focus earlier, e.g. a web browser. It's really annoying.
I also don't understand how still in 2020 when I move a window with the mouse, the window can't keep up with the mouse.
30ms input lag is what we should work towards. But that's not what we have today. Today is actually crap.