31–40 of 142 posts
S3 is the simplest CRUD app you could create. It's essentially just the 4 functions of C.R.U.D done to a file. Most problems in tech are not that simple. Note: not knocking the service. just pointing out not all things are so inherently basic (and valuable at the same time).
S3 is up there as one of my favorite tech products ever. Over the years I've used it for all sorts of things but most recently I've been using it to roll my own DB backup system. One of the things that shocks me about the system is the level of object durability. A few years ago I was taking an AWS certification course and learned that their durability number means that one can expect to loose data about once every 1…
Over the years working on countless projects I’ve come to realize that the more “easy” something looks to an end user, the more work it took to make it that way. It takes a lot of work to create and polish something to the state where you’d call it graceful, elegant and beautiful.
There are exceptions for sure, but often times hidden under every delightful interface is an iceberg of complexity. When something “just works” you know there was a hell of a lot of effort that went into making it so.
S3 is up there as one of my favorite tech products ever. Over the years I've used it for all sorts of things but most recently I've been using it to roll my own DB backup system. One of the things that shocks me about the system is the level of object durability. A few years ago I was taking an AWS certification course and learned that their durability number means that one can expect to loose data about once every 1…
> their durability number means that one can expect to loose data about once every 10,000 years What does that mean? If I have 1 million objects, I loose 100 per year?
If you have ten million objects, you should lose one every 10k years or so.
tf is that title supposed to mean?
Table stakes is a poker term meaning the absolute minimum amount you are allowed to bet. So the title translates to "In S3, simplicity is the bare minimum" or "In S3, simplicity is so important that if we didn't have it, we might as well not even have S3."
Earlier quoted context omitted.
That's going to depend a lot on what your needs are, particularly in terms of redundancy and durability. S3 takes care of a lot of that for you. One server with a RAID array can survive, usually, 1 or maybe 2 drive failures. The remaining drives in the array will have to do more work when a failed drive is replaced and data is copied to the new array member. This sometimes leads to additional failures before replacem…
How many businesses or applications really need 99.999999999% durability and 99.99% availability? Is your whole stack organized to deliver the forementioned durability and availability?
S3 is the simplest CRUD app you could create. It's essentially just the 4 functions of C.R.U.D done to a file. Most problems in tech are not that simple. Note: not knocking the service. just pointing out not all things are so inherently basic (and valuable at the same time).
No problem. I’m sure ChatGPT could cook up a replacement in a weekend. Like Dropbox it’s just rsync with some scripts that glue it together. How hard could it possibly be?
I mean people serve entire websites right out of s3 buckets. Using it as a crude CDN of sorts.
It’s a modern marvel.
S3 is up there as one of my favorite tech products ever. Over the years I've used it for all sorts of things but most recently I've been using it to roll my own DB backup system. One of the things that shocks me about the system is the level of object durability. A few years ago I was taking an AWS certification course and learned that their durability number means that one can expect to loose data about once every 1…
It’s designed for that level of durability, but it’s only as good as a single change or correlated set of hardware failures that can quickly change the theoretical durability model. Or even corrupting data is possible too.
Earlier quoted context omitted.
> their durability number means that one can expect to loose data about once every 10,000 years What does that mean? If I have 1 million objects, I loose 100 per year?
Amazon claims 99.999999999% durability. If you have ten million objects, you should lose one every 10k years or so.
I have a feeling that economies of scale have a point of diminishing returns. At what point does it become more costly and complicated to store your data on S3 versus just maintaining a server with RAID disks somewhere? S3 is an engineering marvel, but it's an insanely complicated backend architecture just to store some files.
That's going to depend a lot on what your needs are, particularly in terms of redundancy and durability. S3 takes care of a lot of that for you. One server with a RAID array can survive, usually, 1 or maybe 2 drive failures. The remaining drives in the array will have to do more work when a failed drive is replaced and data is copied to the new array member. This sometimes leads to additional failures before replacem…
Standard RAID configurations can only handle 2 failures, but there are libraries and filesystems allowing arbitrarily high redundancy.