101–110 of 172 posts
> tens of millions of disks If we assume enterprise HDDs in the double digit TB range then one can estimate that the total S3 storage volume of AWS is in the triple digit Exabyte range. That's propably the biggest storage system on planet earth.
Production scale enterprise HDDs are in the 30TB range, 50TB on the horizon...
Earlier quoted context omitted.
It's assumed that the new S3 Express One Zone is backed by SSDs but I believe Amazon doesn't say so explicitly.
I've always felt it's probably a wrapper around the Amazon EFS due to the similar pricing and that S3 One Zone has "Directory" buckets, a very file system-y idea.
Earlier quoted context omitted.
My own assumption was always that the cold tiers are managed by a tape robot , but managing offlined HDDs rather than actual tapes.
I think that's close to the truth. IIRC it's something like a massive cluster of machines that are effectively powered off 99% of the time with a careful sharding scheme where they're turned on and off in batches over a long period of time for periodic backup or restore of blobs.
So is any of S3 powered by SSD's? I honestly figured that it must be powered by SSD for the standard tier and the slower tiers were the ones using HDD or slower systems.
It is interesting that even after falling prices of HDDs, S3 costs have remained the same for at least 8 years. There's just not enough competition to push them to reduce costs. But imagine money it brings in in AWS because of this.
EDIT: clarity on monthly pricing.
It is interesting that even after falling prices of HDDs, S3 costs have remained the same for at least 8 years. There's just not enough competition to push them to reduce costs. But imagine money it brings in in AWS because of this.
Earlier quoted context omitted.
I dunno, the article's tl;dr is just parallelism. Data gets split into redundant copies, and is rebalanced in response to hot spots. Everything in this article is the obvious answer you'd expect.
It's not really "redundant copies". It's erasure coding (ie, your data is the solution of an overdetermined system of equations).
It is interesting that even after falling prices of HDDs, S3 costs have remained the same for at least 8 years. There's just not enough competition to push them to reduce costs. But imagine money it brings in in AWS because of this.
Does anyone know what is the technology stack of S3? Monolith or multiple services? I assume would have lots of queues, caches and long running workers.