41–50 of 142 posts
Earlier quoted context omitted.
Amazon claims 99.999999999% durability. If you have ten million objects, you should lose one every 10k years or so.
How does that compare to competitors and things like distributed file systems?
Earlier quoted context omitted.
That's going to depend a lot on what your needs are, particularly in terms of redundancy and durability. S3 takes care of a lot of that for you. One server with a RAID array can survive, usually, 1 or maybe 2 drive failures. The remaining drives in the array will have to do more work when a failed drive is replaced and data is copied to the new array member. This sometimes leads to additional failures before replacem…
How many businesses or applications really need 99.999999999% durability and 99.99% availability? Is your whole stack organized to deliver the forementioned durability and availability?
I'll note that data durability is a bit of a different concern than service availability. A service being down for some amount of time sucks, but it'll probably come back up at some point and life moves on. If data is lost completely, it's just gone. It's going to have to be re-created from other sources, generated fresh, or accepted as irreplaceable and lost forever.
Some use cases can tolerate losing some or all of the data. Many can't, so data durability tends to be a concern for non-trivial use cases.
if only metadata could be queried without processing a csv output file first, imagine storing thumbnails in there even! copied objects had actual events, not something you have to dig cloudtrail for, you could get last update time from a bucket to make caching easier
I found out last year that you can actually run a full SPA using S3 and a CDN. It’s kind of a nuts platform
It's always crazy to me that people will run a could be static site on Netlify/Vercel/etc.
S3 is up there as one of my favorite tech products ever. Over the years I've used it for all sorts of things but most recently I've been using it to roll my own DB backup system. One of the things that shocks me about the system is the level of object durability. A few years ago I was taking an AWS certification course and learned that their durability number means that one can expect to loose data about once every 1…
Of course, I lost that script when the server crashed, the one thing I didn't back up properly.
Earlier quoted context omitted.
> their durability number means that one can expect to loose data about once every 10,000 years What does that mean? If I have 1 million objects, I loose 100 per year?
Amazon claims 99.999999999% durability. If you have ten million objects, you should lose one every 10k years or so.
Earlier quoted context omitted.
That's going to depend a lot on what your needs are, particularly in terms of redundancy and durability. S3 takes care of a lot of that for you. One server with a RAID array can survive, usually, 1 or maybe 2 drive failures. The remaining drives in the array will have to do more work when a failed drive is replaced and data is copied to the new array member. This sometimes leads to additional failures before replacem…
> One server with a RAID array can survive, usually, 1 or maybe 2 drive failures. Standard RAID configurations can only handle 2 failures, but there are libraries and filesystems allowing arbitrarily high redundancy.
As long as it's all in one physical location, there's still fire and weather as ways to immediately cut through all that redundancy.
Earlier quoted context omitted.
If you read the article, they say that exactly. Its by Dr Werner Vogels, they know exactly what goes into S3, since they are a principal engineer on the project.
It’s Vogels’ blog but this is a guest post by Andy Warfield.