There are two types of websites, those that have suffered downtime, and those that will.
Amazon was Down
71–80 of 103 posts
Re: Amazon was Down
#72Earlier quoted context omitted.
Does Amazon not run their website on AWS? I assumed (incorrectly, apparently) that AWS was originally built to allow Amazon to scale their own services. Is it really a separate product that they don't use themselves?
Just because Amazon is down, doesn't mean the infrastructure is the reason. They did build AWS out of the technology they used to build Amazon, but its unclear if they are using it directly or use an isolated set of services.
Amazon uses AWS to host Amazon.com (retail), although you can imagine that they dodge the billing structure and have quite a number of resources dedicated away from the main AWS fleets.
Re: Amazon was Down
#73At an estimated loss of $31,000 per minute http://news.cnet.com/8301-10784_3-9962010-7.html?tag=nefd.to... I'm blown away that I see Amazon goes down so often. That certainly, in my mind, doesn't bode well for the brand of AWS.
Amazon.com retail website does not run on AWS.
Amazon.com website actually runs on AWS for last 2-3 years.
Re: Amazon was Down
#74A while ago, someone claiming to have worked at Amazon said that downtime doesn't really affect things as much as you'd think. He said most people simply just come back later. https://news.ycombinator.com/item?id=5147461 [Edit: That being said, there's also the statistic that every 100ms of latency costs Amazon 1%. Imagine what 20+ minutes of "latency" would do. https://news.ycombinator.com/item?id=273900 ]
I imagine the difference between latency and downtime is that latency tends to occur every time you visit, while by definition, downtime is more rare. In other words, latency provides a bad experience, while downtime provides no experience.
Re: Amazon was Down
#75Somewhat off-topic: my (limited) experience with Amazon Prime video suggests it's significantly less reliable than Netflix or iTunes (neither of which are stupendously reliable, but I'd say Netflix is by far the most reliable of the three). Hulu might actually be worse than Amazon Prime.
I don't know if you saw this posted on HN, but Netflix test their system really well. They use so-called chaos monkey [1] that shuts down random servers on a whim. This allows them to detect and get rid of dependencies, i.e. tolerate failures in other parts of the system. [1] http://techblog.netflix.com/2011/07/netflix-simian-army.html
Wish I had thought of chaos monkey, much cooler than whatever I called my version of it.
A few years ago I built an automated test system in perl, complete with message bus and message listener container for running tasks on various servers. One of the automated tests I wrote had a component that would periodically (at random intervals) kill processes, unmount shared filesystems, offline interfaces, etc. to cause failovers, to verify that all processes and resources were failed over, and all tasks were reassigned to other nodes and no jobs were dropped or stalled.
It is really the only way to ensure you've covered your bases - beating the shit out of your system repetitively. It uncovered a bunch of big holes and some very obscure ones too, and once we got those fixed it ran pretty much flawlessly.
Re: Amazon was Down
#76This is just a small complaint coming many hours after this link was posted. You linked to perhaps the most top-level URL amazon has available for a temporary outage. This means a couple things. 1) Hours later, the outage is over and I'm just hitting the home page. No specific information about what you were reporting. 2) This specific link is, as far as I know, now no linger available for other stories. That may not matter I the long run but it bares mentioning.
Re: Amazon was Down
#77At an estimated loss of $31,000 per minute http://news.cnet.com/8301-10784_3-9962010-7.html?tag=nefd.to... I'm blown away that I see Amazon goes down so often. That certainly, in my mind, doesn't bode well for the brand of AWS.
Does Amazon really go down that often? Is there any data how often/ for how long Amazon does go down? I wonder how it compares to other sites that get the same amount of traffic.
Re: Amazon was Down
#78Earlier quoted context omitted.
$31K in '08. What is it today?
A very quick calculation (using AMZN's $61b net sales in 2012) yields about $116k per minute.
One thing that would contribute to extra cost is a large amount of advertising that they are paying per click that ends up leading to a page being down.
Re: Amazon was Down
#79At an estimated loss of $31,000 per minute http://news.cnet.com/8301-10784_3-9962010-7.html?tag=nefd.to... I'm blown away that I see Amazon goes down so often. That certainly, in my mind, doesn't bode well for the brand of AWS.
Heroku is up: http://cl.ly/image/0B0U1K3Z342R . Conversely, when AWS had issues, Amazon.com was not impacted. Amazon.com != AWS. I'm curious to know when AWS or Amazon.com innovations impact each other, or which one leads. I'd rather it be Amazon.com.
Apparently, Amazon.com switched to AWS in 2011 http://www.quora.com/Amazon/Does-Amazon-com-use-Amazon-AWS.
Re: Amazon was Down
#80A while ago, someone claiming to have worked at Amazon said that downtime doesn't really affect things as much as you'd think. He said most people simply just come back later. https://news.ycombinator.com/item?id=5147461 [Edit: That being said, there's also the statistic that every 100ms of latency costs Amazon 1%. Imagine what 20+ minutes of "latency" would do. https://news.ycombinator.com/item?id=273900 ]
>A while ago, someone claiming to have worked at Amazon said that downtime doesn't really affect things as much as you'd think. He said most people simply just come back later.
Brick and mortar stores found that out ages ago. They were closed for large parts of the day/night and the customers just came back the next day.
If people didn't leave Tumblr and Twitter, with their constant massive outages (at some point in their life), when why would the leave Amazon, a huge established player, for a few hours outage?