Earlier quoted context omitted.
SHUT. DOWN. EVERYTHING. Good plan though. :p
My original comment was "We're going Madagascar on the servers." Then I remembered I was on HN, not reddit. :)
You don't need to flee the country yet.
Amazon Web Services are down
211–220 of 346 posts
Re: Amazon Web Services are down
#212Thankfully, my major clients are using the Asia EC2 Instances!
Re: Amazon Web Services are down
#213Earlier quoted context omitted.
"Resolving" probably isn't the right word seeing as this is a purely amazon issue, not much they can do. I'm guessing jedberg is mostly banging his head in walls and seriously looking at alternative hosting solutions right now.
Actaully, I'm on the couch in front of the fireplace, watching old SNL, waiting patiently for Amazon to fix their shit, and figuring out how we can not use EBS anymore.
Right on.
Periodically, latest a couple of days ago, there's a post / discussion about whether outsourcing core functionality is the right thing to do. There are valid points on both sides of the issue.
For my part, if I'm going to be up in the middle of the night I'd rather be up working on fixing something rather than up fretting and checking status. But either way things get fixed. The real difference comes in the following days and weeks. When core stuff is in the cloud then you can try to get assurances and such, fwiw. When core stuff is in-house then you spend time, energy and money making sure you can sleep at night.
Re: Amazon Web Services are down
#214Ruh Roh. The service I'm using to acquire accommodation seems to be dependent on AWS. Guess I'm going to be homeless tomorrow if it doesn't get fixed. :X Note to self. Don't ever build a service reliant on AWS.
If its us (crashpadder.com) - sorry for the inconvenience. Email us hello [at] crashpadder.com and we'll do all we can when we're back up.
We'd moved to AWS from an even worse host a few months ago, and until this morning had been pretty impressed...
Re: Amazon Web Services are down
#215So when big sites deal use Amazon Web Services for major traffic, do they get a serious customer relationship? Or is it just generic email/web support and a status page?
There are subscriptions for various levels of support, from $50/mo for 12 hour response time to $15k/mo for 15 minute response time. http://aws.amazon.com/premiumsupport/
Interesting pricing. The Platinum seems priced to have no one use it, considering how much of a jump it is over Gold.
Re: Amazon Web Services are down
#216Amazon's EC2 SLA is extremely clear - a given region has an availability of 99.95%. If you're running a website and you haven't deployed across across more than one region then, by definition, your website will have 99.95% availailbility. If you want a higher level of availability use more than one region. Amazon's EBS SLA is less clear, but they state that they expect an annual failure rate of 0.1-0.5%, compared to…
Amazon's EC2 SLA is extremely clear -
a given region has an availability of 99.95%.
If you're running a website and you haven't
deployed across across more than one region then,
by definition, your website will have 99.95%
availailbility. If you want a higher level of
availability use more than one region.
let P(region fails) = 0.05% and let's assume (and hope) that the probability of failure of one region is independent of the state of the other regions.
P(two regions fail) = P(one region fails and another region fails) = P(region fails) * P(region fails) = 0.05% * 0.05% = 0.0025%
Making your availability = 100% - 0.0025% = 99.9975%
Ultimately it's more of a business decision if you want to pay for the extra 0.0475% of availability. I would think (or hope) that most engineers would want it anyway.
The numbers at this size appear insignificant. How would one (say an engineer) convince "the management" that the extra 0.0475% of availability is worth the investment/expense?
Re: Amazon Web Services are down
#217Earlier quoted context omitted.
Yay, cloud.
Please do not make content-free posts such as this. It adds no value to the conversation and is only noise. If you actually wish to make a useful point about the practicality or otherwise of massively virtualised systems for webapp deployment, please do. It's going to take more than two words though.
Please do not make content-free posts such as this. It adds no value to the conversation and is only noise.
If you actually wish to bitch about a post that we all got the point of, you're going to need more than two paragraphs.
Re: Amazon Web Services are down
#218Yay for relying on the cloud \o/
What do you do when a building contractor cuts through your datacentre's link? For example, this happened in London a while back. My point is: things go wrong. Blaming this on the cloud is silly.
It happened to ServInt back in 2004 - the VA Department of Transportation cut the MCI OC-48 and OC-12 fiber
strands running to the ServInt datacenter. ServInt handled it amazingly well and learned from their lessons moving forward and if I recall correctly gave a comp'd month to it's customers. They followed up by adding additional backbones in addition to the MCI one and have always been solid since.
Re: Amazon Web Services are down
#219Earlier quoted context omitted.
> Wouldn't that be something to light a fire under the collective arses of those in charge of keeping EBS stable. Yeah, the only possible explanation is that Amazon's put a bunch of lazy guys in charge of EBS. How hard a job could it be?
I'm certain they're plenty talented and hard-working, but there's got to be something in their way. Just like the guy said as he was leaving Reddit, management seems to be pointing fingers in different directions as to why this is going on and how it can be resolved. It's sad, though, because I don't want to think of Amazon as a company with a potentially toxic corporate culture. I enjoy my free Prime membership as a…
Amazon's corporate culture works pretty well. The problem is that these large-scale multi-tenant services are new technology and very difficult to create and run.