Live data from Hacker News

Amazon S3 – 2 Trillion Objects, 1.1 Million Requests/Second

aws.typepad.com

21–30 of 63 posts

Re: Amazon S3 – 2 Trillion Objects, 1.1 Million Requests/Second

#21

Anyone else find this surprisingly low? I'd imagine your typical web service holds a few thousand objects in S3 for images etc, then backups and anything else. Then you have your big players like netflix, dropbox etc that use the service. Who store data for tens of millions of customers...

1.1 Million Requests/Second seems especially low for Amazon.

The average server can serve more than 1,000 static objects / second easily.

Re: Amazon S3 – 2 Trillion Objects, 1.1 Million Requests/Second

#23
"If you added one S3 object every 60 hours starting at the Big Bang, you'd have accumulated almost two trillion of them by now."

That actually sounds underwhelming! IMHO our brains have an easier time thinking "hey, 1 every 60 hours that's not much" compared to figuring out the universe is really incredibly old ;-)

Re: Amazon S3 – 2 Trillion Objects, 1.1 Million Requests/Second

#25
post #10
post #3

It would be really interesting to know the average size of an object and visualize the amount of harddisks it takes to store all this data.

To put this into perspective it's 2.7B new objects per day (assuming 1 Trillion objects averaged over 365 days). Assuming each object is 100KB (generous estimate, after compression) that would be 270GB per day -- or assuming ten levels of redundancy and striped across three RAID storage devices (per level of redundancy) then 8.1TB per day. I'm not familiar with their hard disk procurement policies but it wouldn't be…

I'm not convinced that 100KB is a great estimate on file size, but either way you're off by a few zeroes. It's not 270GB per day, it's 270TB. Even if each object were just one byte, that would be 2.7GB. 100KB is one hundred thousand bytes. So it's quite a bit more than eleven drives per day!

Re: Amazon S3 – 2 Trillion Objects, 1.1 Million Requests/Second

#26

"If you added one S3 object every 60 hours starting at the Big Bang, you'd have accumulated almost two trillion of them by now." That actually sounds underwhelming! IMHO our brains have an easier time thinking "hey, 1 every 60 hours that's not much" compared to figuring out the universe is really incredibly old ;-)

I'm open to more creative analogies. I'll happily send some AWS stickers to the first 5 truly great ones that show up under this reply...

Re: Amazon S3 – 2 Trillion Objects, 1.1 Million Requests/Second

#27
post #10
post #3

It would be really interesting to know the average size of an object and visualize the amount of harddisks it takes to store all this data.

To put this into perspective it's 2.7B new objects per day (assuming 1 Trillion objects averaged over 365 days). Assuming each object is 100KB (generous estimate, after compression) that would be 270GB per day -- or assuming ten levels of redundancy and striped across three RAID storage devices (per level of redundancy) then 8.1TB per day. I'm not familiar with their hard disk procurement policies but it wouldn't be…

[deleted]

Re: Amazon S3 – 2 Trillion Objects, 1.1 Million Requests/Second

#28

"If you added one S3 object every 60 hours starting at the Big Bang, you'd have accumulated almost two trillion of them by now." That actually sounds underwhelming! IMHO our brains have an easier time thinking "hey, 1 every 60 hours that's not much" compared to figuring out the universe is really incredibly old ;-)

I'm open to more creative analogies. I'll happily send some AWS stickers to the first 5 truly great ones that show up under this reply...

Awesome ;-)

How about comparing to the lifetime of one person? With the US average life expectancy (78 years) that means you'd have to upload 800 objects/second for your whole life to get to 2 trillon :)

Edit: One S3 object for each fish in the ocean (3 to 4 trillion) could also be a nice future milestone (if also slightly underwhelming :)

Edit2: I also love the eye blink as a unit of time. Each time you blink (average is around 15 times a minute), XXX more objects will have been uploaded to/requested on S3

Re: Amazon S3 – 2 Trillion Objects, 1.1 Million Requests/Second

#29

"If you added one S3 object every 60 hours starting at the Big Bang, you'd have accumulated almost two trillion of them by now." That actually sounds underwhelming! IMHO our brains have an easier time thinking "hey, 1 every 60 hours that's not much" compared to figuring out the universe is really incredibly old ;-)

I'm open to more creative analogies. I'll happily send some AWS stickers to the first 5 truly great ones that show up under this reply...

If each object was written on a single standard 8 1/2 x 11 inch piece of paper and stacked up, the stack would be about 125,000 miles high (about half the distance to the moon).

Re: Amazon S3 – 2 Trillion Objects, 1.1 Million Requests/Second

#30

"If you added one S3 object every 60 hours starting at the Big Bang, you'd have accumulated almost two trillion of them by now." That actually sounds underwhelming! IMHO our brains have an easier time thinking "hey, 1 every 60 hours that's not much" compared to figuring out the universe is really incredibly old ;-)

I'm open to more creative analogies. I'll happily send some AWS stickers to the first 5 truly great ones that show up under this reply...

Imagine each object being a person, and imagine a room large enough to fit 10 000 people, the size of a small village, standing side by side, barely being able able to touch each other's hands, spanning roughly 15-20 km.

Now imagine putting 9 999 people behind every person.

And finally, imagine stacking 9 999 on each of these million people's head. That's twice the height a normal airplane flies at.

Then double that number. That's how many objects have been created so far.

---

A bit long, but I think the roughly human scale numbers, 999s and even more effect makes it almost imaginable.

EDIT: Another one - you could fill the whole of manhattan with cents and still have money to spare.

(Manhattan is 87.46 km^2 and a coin is less than about 40mm^2)

Post reply on HN