Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Web Services are down

status.aws.amazon.com

Post reply on HN
Print view View original

231–240 of 346 posts

Re: Amazon Web Services are down

#231

Post by bartonfink »

hoop wrote: post #216
asymptotic wrote: post #129

Amazon's EC2 SLA is extremely clear - a given region has an availability of 99.95%. If you're running a website and you haven't deployed across across more than one region then, by definition, your website will have 99.95% availailbility. If you want a higher level of availability use more than one region. Amazon's EBS SLA is less clear, but they state that they expect an annual failure rate of 0.1-0.5%, compared to…

Amazon's EC2 SLA is extremely clear - a given region has an availability of 99.95%. If you're running a website and you haven't deployed across across more than one region then, by definition, your website will have 99.95% availailbility. If you want a higher level of availability use more than one region. Good point. let P(region fails) = 0.05% and let's assume (and hope) that the probability of failure of one regio…

Why wouldn't a simple expected value calculation work? You've shown that you can calculate the extra availability that subscribing to another region provides. Simply multiply the cost of an outage by the extra availability provided by an additional region that would have prevented that outage.

If expanding to another region costs more than just taking the outage, then it's categorically not a good option. If management still says no in the face of numbers that suggest yes, then that tells you that you're missing a hidden objection, and how you proceed will depend on a lot of factors specific to your situation.

Re: Amazon Web Services are down

#232

Post by cooldeal »

JonnieCache wrote: post #207
cooldeal wrote: post #189

Earlier quoted context omitted.

Yay, cloud.

Please do not make content-free posts such as this. It adds no value to the conversation and is only noise. If you actually wish to make a useful point about the practicality or otherwise of massively virtualised systems for webapp deployment, please do. It's going to take more than two words though.

I guess you haven't seen the Microsoft ads about the cloud? http://www.youtube.com/watch?v=Lel3swo4RMc

Anyway, my bad, I was just trying to make a joke to lighten up the mood. Sorry.

Re: Amazon Web Services are down

#233

Post by astrodust »

G_Morgan wrote: post #130
hendi_ wrote: post #16

Yay for relying on the cloud \o/

It isn't about relying on the cloud. It is more about incompetent cloud providers.

Does seem harsh to brand Amazon as incompetent. Things like this probably happen to Google's server farm all the time, but since they're the only customer they can and do reconfigure everything massively to adapt.

Amazon can't do that to your application that runs on a handful of systems.

Re: Amazon Web Services are down

#234

Post by [deleted] »

jedberg wrote: post #15
kylec wrote: post #5

I guess this is one Reddit outage that can't be blamed on poor scaling

Thankfully, no. :) But yeah, right now we're shutting everything down to try and avoid possible data corruption. Once they restore service, hopefully we'll be able to come back quickly.

[deleted]

Re: Amazon Web Services are down

#236

Post by blhack »

stef25 wrote: post #132
asymptotic wrote: post #129

Amazon's EC2 SLA is extremely clear - a given region has an availability of 99.95%. If you're running a website and you haven't deployed across across more than one region then, by definition, your website will have 99.95% availailbility. If you want a higher level of availability use more than one region. Amazon's EBS SLA is less clear, but they state that they expect an annual failure rate of 0.1-0.5%, compared to…

Reddit's been down for several hours today, I'm sure they are already way lower 99.95%.

http://cache-scale.appspot.com/c/www.reddit.com/

That's a cache of it. I really wish that the admins at reddit would implement something like this themselves, then link to it when downtime like this happens.

Re: Amazon Web Services are down

#237

Post by recoiledsnake »

JonnieCache wrote: post #207
cooldeal wrote: post #189

Earlier quoted context omitted.

Yay, cloud.

Please do not make content-free posts such as this. It adds no value to the conversation and is only noise. If you actually wish to make a useful point about the practicality or otherwise of massively virtualised systems for webapp deployment, please do. It's going to take more than two words though.

I laughed, it's relevant and puts things in perspective if you had seen the ads. So it's not really content free even if it's just two words.

Re: Amazon Web Services are down

#238

Post by jedberg »

bvi wrote: post #225
jedberg wrote: post #175

Earlier quoted context omitted.

We rely heavily on EBS still, so this is hurting us more than most others. Hopefully they'll have us back up soon.

You guys might have answered this in one of your AMAs/blog posts (or was it raldi who commented?), but what options can reddit resort to should this stuff happen again to this degree of severity?

We're moving away from the EBS product altogether. The hard part is dealing with the master databases. Normally I'd have a master database with a built in raid-10, but I can't do that on EC2, so I have to come up with another option.

So I guess that is the long way of saying that hopefully it won't happen again.

Re: Amazon Web Services are down

#239

Post by AgentConundrum »

reustle wrote: post #179
jedberg wrote: post #105

Earlier quoted context omitted.

Send me the link to your ad (when we come back up) and I'll comp you a day.

Isn't it paid on impressions?

I don't know how the sidebar ads work, but the featured links at the top of the page are run on a sort of auction system. Everybody that wants a piece pays however much they want (minimum of $20) for the day, then all the ads are totaled up and each ad gets a percentage of pageviews corresponding to the their percentage paid on that days revenues.

If that wasn't clear (and I'm not sure it was), assume you and I were both the only advertisers on reddit for a day. If I pay $20 and you pay $20, we would both have our ads displayed on 50% of pageviews for that day. If instead I paid $80 and you still only paid $20, then I would get 80% of pageviews to your 20%, regardless of the total pageviews for the day.

Re: Amazon Web Services are down

#240

Post by jedberg »

campnic wrote: post #223
timf wrote: post #222

Some quotes regarding how Netflix handled this without interruptions: "Netflix showed some increased latency, internal alarms went off but hasn't had a service outage." [1] "Netflix is deployed in three zones, sized to lose one and keep going. Cheaper than cost of being down." [2] [1] https://twitter.com/adrianco/status/61075904847282177 [2] https://twitter.com/adrianco/status/61076362680745984

"Cheaper than cost of being down." This is very insightful. Many of us look at the cost of multi zone deployments and cringe, but its a mathematics exercise. (.05 * hours in a year)*(cost of being down per hour) = (expected cost of single zone availability). Now just compare to 2-3x your single zone deployment cost. Don't forget the cost of being down per hour should include lost customers as well.

At their level of income, this is true.

For us, we are just now staffing up to the level where we can make the changes necessary to do the same thing.

Post reply on HN