Live data from Hacker News

Amazon Web Services are down

status.aws.amazon.com

31–40 of 346 posts

Re: Amazon Web Services are down

#32
post #2

Current status: bad things are happening in the North Virginia datacenter. EC2, EBS and RDS are all down on US-east-1. Edit: Heroku, Foursquare, Quora and Reddit are all experiencing subsequent issues.

Not all EC2 & EBS instances are down. I have several in US-east-1a and 1 is down, while all of the others are working.

Re: Amazon Web Services are down

#34

I think this is a good example of how the "cloud" is not a silver bullet to making your site always up. AWS provides a way to keep it up, but it is up to each developer to ensure that they are using AWS in a way to make sure their site can handle problems in one availability zone. I think we will see more of a focus from big users of AWS about focusing on how to create a redundant service using AWS. Or at least I hop…

This outage is affecting all AZ's in the East. So even a multizone setup wouldn't help for this one. Only a multiregion setup.

This outage is a lot like having your entire datacenter lose power.

Re: Amazon Web Services are down

#36

I think this is a good example of how the "cloud" is not a silver bullet to making your site always up. AWS provides a way to keep it up, but it is up to each developer to ensure that they are using AWS in a way to make sure their site can handle problems in one availability zone. I think we will see more of a focus from big users of AWS about focusing on how to create a redundant service using AWS. Or at least I hop…

All well and good, but the elephant in the room is that multiple availability zones have failed at the same time. It looks like AWS have a single point of failure they weren't previously aware of.

Re: Amazon Web Services are down

#37
post #32
post #2

Current status: bad things are happening in the North Virginia datacenter. EC2, EBS and RDS are all down on US-east-1. Edit: Heroku, Foursquare, Quora and Reddit are all experiencing subsequent issues.

Not all EC2 & EBS instances are down. I have several in US-east-1a and 1 is down, while all of the others are working.

Same with us. About 10% of our 700+ volumes are having problems right now.

It's hard to tell for sure since there isn't any load.

Re: Amazon Web Services are down

#38
post #8

Earlier quoted context omitted.

Apparently most of their problems are caused by bad EBS writes/performance, or at least so they said a few weeks ago after some particularly bad downtime. It looks like EBS will randomly decide to switch to a few bps of performance from time to time. I would use Amazon for my startup, but these issues really make it hard to justify.

Slightly offtopic, but wasn't that post by an ex-employee as a comment? Not that the technical aspect of it wasn't fantastic, because it was, but I don't think Reddit said anything publically did they?

It was, you are right, I misremembered. valisystem's comments above contains the reference.

Re: Amazon Web Services are down

#40
post #6

Earlier quoted context omitted.

Looks like troubles in only one availability zone.

AWS have now confirmed that this affects multiple availability zones. From the status page: "..impacting EBS volumes in multiple availability zones in the US-EAST-1 region"

Thats not good. The whole point of multiple AVs is for them to not fail at the same time. Suggests some dependencies that should not be there perhaps, or at least some correlation of something, like software upgrades. Looking for a good explanation of this; one AV going down is not a problem and should not impact anyone who is load balancing.
Post reply on HN