Earlier quoted context omitted.
http://status.heroku.com/ for those on Heroku
Is it just me, or is their status page down?
I know you probably have your answer by now, but this site always helps me out when I have that question:
Amazon Web Services are down
221–230 of 346 posts
Re: Amazon Web Services are down
#222Some quotes regarding how Netflix handled this without interruptions:
"Netflix showed some increased latency, internal alarms went off but hasn't had a service outage." [1]
"Netflix is deployed in three zones, sized to lose one and keep going. Cheaper than cost of being down." [2]
Re: Amazon Web Services are down
#223Some quotes regarding how Netflix handled this without interruptions: "Netflix showed some increased latency, internal alarms went off but hasn't had a service outage." [1] "Netflix is deployed in three zones, sized to lose one and keep going. Cheaper than cost of being down." [2] [1] https://twitter.com/adrianco/status/61075904847282177 [2] https://twitter.com/adrianco/status/61076362680745984
"Cheaper than cost of being down."
This is very insightful. Many of us look at the cost of multi zone deployments and cringe, but its a mathematics exercise.
(.05 * hours in a year)*(cost of being down per hour) = (expected cost of single zone availability).
Now just compare to 2-3x your single zone deployment cost. Don't forget the cost of being down per hour should include lost customers as well.
Re: Amazon Web Services are down
#224Re: Amazon Web Services are down
#225Earlier quoted context omitted.
Foursquare is up now. Quora not showing the 503 anymore, Reddit still down.
We rely heavily on EBS still, so this is hurting us more than most others. Hopefully they'll have us back up soon.
You guys might have answered this in one of your AMAs/blog posts (or was it raldi who commented?), but what options can reddit resort to should this stuff happen again to this degree of severity?
Re: Amazon Web Services are down
#226Earlier quoted context omitted.
This outage is affecting all AZ's in the East. So even a multizone setup wouldn't help for this one. Only a multiregion setup. This outage is a lot like having your entire datacenter lose power.
If "multi-region" means North America, Europe and Asia Pacific, doing so would also improve world-wide latency (e.g. here in Australia...). Could you use this outage to justify switching to multi-region?
I don't know reddit, but concurrency is very hard with high latency.
Re: Amazon Web Services are down
#227Earlier quoted context omitted.
Reddit's been down for several hours today, I'm sure they are already way lower 99.95%.
0.05% of one year is 4 hours, 22 minutes and 48 seconds.
Is this really standard practice for measuring the SLA? The contracts I've seen for a couple small businesses are generally per billable period.
Which always made sense to me. I pay you for 99.5% uptime this month. If you don't achieve it, then I get a discount, as simple as that. If your availability is below that, I don't pay full price for the billable period and then reconcile at the end of the year.
Any links or general advice on this topic anyone has I'd be pretty interested on finding out if there's a general consensus of it being done differently?
Re: Amazon Web Services are down
#228Amazon's EC2 SLA is extremely clear - a given region has an availability of 99.95%. If you're running a website and you haven't deployed across across more than one region then, by definition, your website will have 99.95% availailbility. If you want a higher level of availability use more than one region. Amazon's EBS SLA is less clear, but they state that they expect an annual failure rate of 0.1-0.5%, compared to…
Amazon's EC2 SLA is extremely clear - a given region has an availability of 99.95%. If you're running a website and you haven't deployed across across more than one region then, by definition, your website will have 99.95% availailbility. If you want a higher level of availability use more than one region. Good point. let P(region fails) = 0.05% and let's assume (and hope) that the probability of failure of one regio…
> The numbers at this size appear insignificant. How would one (say an engineer) convince "the management" that the extra 0.0475% of availability is worth the investment/expense?
In businesses where physical goods are sold to customers, "the management" is generally very motivated to avoid stock-out situations in which sales are lost due to lack of inventory (even if it's only a very small percentage.) The reason for this is because they are concerned about the potential loss of customer goodwill. It seems that the same applies in this situation.
Re: Amazon Web Services are down
#229Re: Amazon Web Services are down
#230Earlier quoted context omitted.
Amazon is really not being kind to you guys; I sort of hope you'll find an alternative solution fast!
If I was Rackspace, I'd be at Reddit/Wired's headquarters already.
Didn't Jedburg say that they could reduce the failure by spending with Amazon.
I wonder if Rackspace really want this particular traffic burden. It seems that if Reddit choose not to pay for the load they need then you get lot's of bad press for it ... perhaps I'm seeing it wrong.
Rubbish analogy: Kinda like if I was doing a haulage business and you called out for a wheelbarrow to carry some elephants, then when the barrows broke we got bad press despite. If you'd paid for a heavy animal transport package ... OK it's all going wrong, you get the idea.