[deleted]
In S3 simplicity is table stakes
21–30 of 142 posts
Re: In S3 simplicity is table stakes
#22I have a feeling that economies of scale have a point of diminishing returns. At what point does it become more costly and complicated to store your data on S3 versus just maintaining a server with RAID disks somewhere? S3 is an engineering marvel, but it's an insanely complicated backend architecture just to store some files.
That's going to depend a lot on what your needs are, particularly in terms of redundancy and durability. S3 takes care of a lot of that for you.
One server with a RAID array can survive, usually, 1 or maybe 2 drive failures. The remaining drives in the array will have to do more work when a failed drive is replaced and data is copied to the new array member. This sometimes leads to additional failures before replacement completes, because all the drives in the array are probably all the same model bought at the same time and thus have similar manufacturing quality and materials. This is part of why it's generally said that RAID != backup.
You can make a local backup to something like another server with its own storage, external drives, or tape storage. Capacity, recovery time, and cost varies a lot across the available options here. Now you're protected against the original server failing, but you're not protected against location-based impacts - power/network outages, weather damage/flooding, fire, etc.
You can make a remote backup. That can be in a location you own / control, or you can pay someone else to use their storage.
Each layer of redundancy adds cost and complexity.
AWS says they can guarantee 99.999999999% durability and 99.99% availability. You can absolutely design your own system that meets those thresholds, but that is far beyond what one server with a RAID array can do.
Re: In S3 simplicity is table stakes
#23tf is that title supposed to mean?
Somehow gambling is directly correlated to using S3? That’s the best I got.
Re: In S3 simplicity is table stakes
#24tf is that title supposed to mean?
Table stakes is a poker term meaning the absolute minimum amount you are allowed to bet. So the title translates to "In S3, simplicity is the bare minimum" or "In S3, simplicity is so important that if we didn't have it, we might as well not even have S3."
Re: In S3 simplicity is table stakes
#25S3 is up there as one of my favorite tech products ever. Over the years I've used it for all sorts of things but most recently I've been using it to roll my own DB backup system. One of the things that shocks me about the system is the level of object durability. A few years ago I was taking an AWS certification course and learned that their durability number means that one can expect to loose data about once every 1…
> their durability number means that one can expect to loose data about once every 10,000 years
What does that mean? If I have 1 million objects, I loose 100 per year?
Re: In S3 simplicity is table stakes
#26S3 is the simplest CRUD app you could create.
It's essentially just the 4 functions of C.R.U.D done to a file.
Most problems in tech are not that simple.
Note: not knocking the service. just pointing out not all things are so inherently basic (and valuable at the same time).
Re: In S3 simplicity is table stakes
#27S3 is the simplest CRUD app you could create. It's essentially just the 4 functions of C.R.U.D done to a file. Most problems in tech are not that simple. Note: not knocking the service. just pointing out not all things are so inherently basic (and valuable at the same time).
Isn't that the most impressive part? That the abstraction makes it seem so simple
Re: In S3 simplicity is table stakes
#28S3 is up there as one of my favorite tech products ever. Over the years I've used it for all sorts of things but most recently I've been using it to roll my own DB backup system. One of the things that shocks me about the system is the level of object durability. A few years ago I was taking an AWS certification course and learned that their durability number means that one can expect to loose data about once every 1…
> their durability number means that one can expect to loose data about once every 10,000 years What does that mean? If I have 1 million objects, I loose 100 per year?
What it means is in any given year, you have a 1 in 10,000 chance that a data loss event occurs. It doesn’t stack like that.
If you had light bulbs that lasted 1,000 hrs on average, and you had 10k light bulbs, and turned them all on at once, then they would all last 1,000 hours on average. Some would die earlier and some later, but the top line number does not tell you anything about the distribution, only the average (mean). That’s what MTTF is; the mean time for a given part to where it has a greater likelihood to have failed by then vs not. It doesn’t tell you if the distribution of light bulbs burning out is 10 hrs or 500 hrs wide. it’s the latter, you’ll start seeing bulbs out within 750 hrs, but if the former it’d be 995 hrs before anything burned out.
Re: In S3 simplicity is table stakes
#29[dead]
Re: In S3 simplicity is table stakes
#30I have a feeling that economies of scale have a point of diminishing returns. At what point does it become more costly and complicated to store your data on S3 versus just maintaining a server with RAID disks somewhere? S3 is an engineering marvel, but it's an insanely complicated backend architecture just to store some files.
That's going to depend a lot on what your needs are, particularly in terms of redundancy and durability. S3 takes care of a lot of that for you. One server with a RAID array can survive, usually, 1 or maybe 2 drive failures. The remaining drives in the array will have to do more work when a failed drive is replaced and data is copied to the new array member. This sometimes leads to additional failures before replacem…
How many businesses or applications really need 99.999999999% durability and 99.99% availability?
Is your whole stack organized to deliver the forementioned durability and availability?