Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon S3’s 15th Birthday: 5,475 Days and 100T Objects

aws.amazon.com

Post reply on HN
Print view View original

21–30 of 79 posts

Re: Amazon S3’s 15th Birthday: 5,475 Days and 100T Objects

#21

Post by ZiiS »

TedDoesntTalk wrote: post #4

> S3 is designed to provide “11 9’s” (99.999999999%) of durability Has anyone lost data on S3 or know anyone who has?

If it works as designed I guess 1000 have; but how would you know. Did your perfect code running on an perfect CPU and RAM connected though a perfect network upload the file with 11 9's certainty?

Re: Amazon S3’s 15th Birthday: 5,475 Days and 100T Objects

#22

Post by jliptzin »

leesalminen wrote: post #2

> That’s almost 13,000 objects for each person in the world, or 50 objects for every one of the roughly two trillion galaxies (according to this 2021 estimate) in the Universe. That’s mind boggling to think about. I wonder how much paper would’ve been needed to store 100T objects on paper. It’s like the new Library of Alexandria.

What’s way more mind boggling to me is that there are at least 260 galaxies per person in the world. I am not shocked at all that S3 is home to any number of cat pictures many of which are probably duplicates.

Re: Amazon S3’s 15th Birthday: 5,475 Days and 100T Objects

#25

Post by PaulHoule »

I am worried.

We will start having trouble with random UUID collisions when we have made 2^64 of them, which is a huge number. It's about as many iron atoms as are in an iron filing.

It is also about 200,000 times 100 trillion, so AWS is just 18 doubling times away from the UUID system breaking down.

Re: Amazon S3’s 15th Birthday: 5,475 Days and 100T Objects

#26

Post by globular-toast »

intsunny wrote: post #3

Can we update the title? 100T reads as "100 terabytes" instead of "100 trillion" (as per the source).

It should be read as "100 tera-objects", meaning 100 trillion objects. Writing "K" seems to be so widespread these days (Youtube likes/dislikes are written as 1.5K, for example), I'm surprised there is any confusion here.

Re: Amazon S3’s 15th Birthday: 5,475 Days and 100T Objects

#27

Post by klodolph »

corban1 wrote: post #9

I am really getting afraid of this centralization of data.

Eh… the thing about cloud storage is that it can be efficient in ways that smaller players cannot. A system like Amazon S3 can pack its hard drives to the brim with data and pay a fairly small overhead for replicas & unused drives.

If you try to decentralize the data, or keep your data yourself, you end up with something that uses more resources, has worse durability, or is worse along some other axis.

I am not so worried about centralization yet. There is still healthy competition between different cloud storage vendors. My impression is that cheap storage is used to make other cloud services more attractive—which means that Amazon’s incentives here are somewhat aligned with mine (as a customer). Personally I am much more worried about the centralization of the shopping experience.

Re: Amazon S3’s 15th Birthday: 5,475 Days and 100T Objects

#28

Post by simonebrunozzi »

If you're curious about some "S3 internals", this is a talk I gave in 2009 [0], based on a ton of interesting stuff that S3's then-GM Alyssa Henry [1] shared with me.

[0]: https://vimeo.com/7330740

[1]: http://itc.conversationsnetwork.org/shows/detail5273.html

Re: Amazon S3’s 15th Birthday: 5,475 Days and 100T Objects

#29

Post by rhacker »

TedDoesntTalk wrote: post #4

> S3 is designed to provide “11 9’s” (99.999999999%) of durability Has anyone lost data on S3 or know anyone who has?

I imagine that in most cases it's a giant archive - data that no one really cares about (I mean they do in principal). Then there's probably like 0.1% that's probably operationally important data - which is actually huge.

This is just a guess based on my own usage of it over the last 15 years at different companies.

Post reply on HN