Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How AWS S3 serves 1 petabyte per second on top of slow HDDs

bigdata.2minutestreaming.com

Post reply on HN
Print view View original

1–10 of 172 posts

Re: How AWS S3 serves 1 petabyte per second on top of slow HDDs

#3

Post by crabique »

Is there an open source service designed with HDDs in mind that achieves similar performance? I know none of the big ones work that well with HDDs: MinIO, Swift, Ceph+RadosGW, SeaweedFS; they all suggest flash-only deployments.

Recently I've been looking into Garage and liking the idea of it, but it seems to have a very different design (no EC).

Re: How AWS S3 serves 1 petabyte per second on top of slow HDDs

#5

Post by giancarlostoro »

crabique wrote: post #3

Is there an open source service designed with HDDs in mind that achieves similar performance? I know none of the big ones work that well with HDDs: MinIO, Swift, Ceph+RadosGW, SeaweedFS; they all suggest flash-only deployments. Recently I've been looking into Garage and liking the idea of it, but it seems to have a very different design (no EC).

Doing some light googling aside from Ceph being listed, there's one called Gluster as well. Hypes itself as "using common off-the-shelf hardware you can create large, distributed storage solutions for media streaming, data analysis, and other data- and bandwidth-intensive tasks."

It's open source / free to boot. I have no direct experience with it myself however.

https://www.gluster.org/

Re: How AWS S3 serves 1 petabyte per second on top of slow HDDs

#6

Post by enether »

giancarlostoro wrote: post #4
EwanToo wrote: post #2

I think a more interesting article on S3 is "Building and operating a pretty big storage system called S3" https://www.allthingsdistributed.com/2023/07/building-and-op...

Really nice read, thank you for that.

Author of the 2minutestreaming blog here. Good point! I'll add this as a reference at the end. I loved that piece. My goal was to be more concise and focus on the HDD aspect

Re: How AWS S3 serves 1 petabyte per second on top of slow HDDs

#7

Post by elitepleb »

crabique wrote: post #3

Is there an open source service designed with HDDs in mind that achieves similar performance? I know none of the big ones work that well with HDDs: MinIO, Swift, Ceph+RadosGW, SeaweedFS; they all suggest flash-only deployments. Recently I've been looking into Garage and liking the idea of it, but it seems to have a very different design (no EC).

Any of them will work just as well, but only with many datacenters worth of drives, which very few deployments can target.

It's the classic horizontal/vertical scaling trade off, that's why flash tends to be more space/cost efficient for speedy access.

Re: How AWS S3 serves 1 petabyte per second on top of slow HDDs

#8

Post by bayindirh »

crabique wrote: post #3

Is there an open source service designed with HDDs in mind that achieves similar performance? I know none of the big ones work that well with HDDs: MinIO, Swift, Ceph+RadosGW, SeaweedFS; they all suggest flash-only deployments. Recently I've been looking into Garage and liking the idea of it, but it seems to have a very different design (no EC).

Lustre and ZFS can do similar speeds.

However, if you need high IOPS, you need flash on MDS for Lustre and some Log SSDs (esp. dedicated write and read ones) for ZFS.

Post reply on HN