Systems Correctness Practices at Amazon Web Services • Hacker News Board

Re: Systems Correctness Practices at Amazon Web Services

#141

Post by mrngm » Mon, Jun 02, 2025, 6:07 AM UTC

slt2021 wrote: post #90

agentultra wrote: post #13

Earlier quoted context omitted.

The TigerBeetle team does this too and it's an interesting approach. One of their developers did a talk on it at HYTRADBOI this year [0]. [0] https://www.hytradboi.com/2025/c222d11a-6f4d-4211-a243-f5b7f...

they also made a browser game to showcase their DST, where you can inject faults and see how tigerbeetle recovers https://sim.tigerbeetle.com/

Here's their blog post from 2023 explaining what they did: https://tigerbeetle.com/blog/2023-07-11-we-put-a-distributed...

Re: Systems Correctness Practices at Amazon Web Services

#142

Post by 0xbadcafebee » Mon, Jun 02, 2025, 3:07 PM UTC

positisop wrote: post #68

simonw wrote: post #47

S3 remains one of the most amazing pieces of software I've ever seen. That thing a few years ago where they just added strong read-after-write consistency to the whole system? Incredible software engineering. https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-rea...

Google Cloud Storage had it for eons before S3. GCS comes across as a much better thought-out and built product.

> GCS comes across as a much better thought-out and built product

I've worked with AWS and GCS for a while, and I have the opposite opinion. GCS is what you get if you let engineers dictate to customers how they are allowed to do work, and then give them shit interfaces, poor documentation, and more complexity than adds value.

There's "I engineered the ultimate engineery thing", and then there's "I made something people actually like using".

Re: Systems Correctness Practices at Amazon Web Services

#143

Post by UltraSane » Mon, Jun 02, 2025, 5:49 PM UTC

simonw wrote: post #134

ninetyninenine wrote: post #118

Earlier quoted context omitted.

Red dead redemption 2 is likely on over 74 million hard drives.

I think you misunderstood. They're not saying S3 uses a million hard drives, they're saying that there exist some large single buckets that use a million hard drives just for that one bucket/customer!

actually data from more than one customer would be stored on those million drives. But data from one customer is spread over 1 million drives to get the needed IOPs from spinning hard drives.