Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon Web Services are down

status.aws.amazon.com

Post reply on HN
Print view View original

121–130 of 346 posts

Re: Amazon Web Services are down

#121

Post by G_Morgan »

jedberg wrote: post #104
masklinn wrote: post #77

Earlier quoted context omitted.

"Resolving" probably isn't the right word seeing as this is a purely amazon issue, not much they can do. I'm guessing jedberg is mostly banging his head in walls and seriously looking at alternative hosting solutions right now.

Actaully, I'm on the couch in front of the fireplace, watching old SNL, waiting patiently for Amazon to fix their shit, and figuring out how we can not use EBS anymore.

TBH Amazon is so bad at this point that turning off Reddit is as good as trying to keep it running. Of course then you need to deal with the increased suicide rate.

Re: Amazon Web Services are down

#122

Post by G_Morgan »

athesyn wrote: post #33
jedberg wrote: post #12

Yes, they are. :(

My four-day weekend is already off to a bad start(UK here).

Mine is worse. I booked Tues-Thurs off. I only have internet in work at the moment. I'm going to miss reddit now and be without internet until I return to work on the 3rd of May. Stupid Sky and their stupid take forever switch overs.

Re: Amazon Web Services are down

#123

Post by bad_user »

jwr wrote: post #85

It's really mostly EBS failures, so the title is overly dramatic. And EBS has been known to have issues.

I've got 2 EC2 instances with EBS root devices -- both are unreachable. My RDS instance is responding, but it is very slow -- struggling to make a backup.

Re: Amazon Web Services are down

#124

Post by pvh »

jedberg wrote: post #15
kylec wrote: post #5

I guess this is one Reddit outage that can't be blamed on poor scaling

Thankfully, no. :) But yeah, right now we're shutting everything down to try and avoid possible data corruption. Once they restore service, hopefully we'll be able to come back quickly.

Hey Jedberg, if you guys aren't already rolling your own, check out fdr's WAL-E tool. It bounces postgres write-ahead logs off S3 and goes great with the new PG9 replication.

https://github.com/heroku/WAL-E

Re: Amazon Web Services are down

#125

Post by bad_user »

ghshephard wrote: post #115
olegp wrote: post #101

Assuming the problem is indeed with EBS, I would say this should be a warning sign to anyone considering going with a PaaS provider, which Amazon is quickly becoming, instead of an IaaS provider like Slicehost or Linode. The increased complexity of their offering makes it more likely that things will break, leaving you locked in. I did a 15 minute talk on the subject, which you can check out here: http://iforum.com.u…

Every time someone makes the claim that downtime should be a warning sign about going with a PaaS provider (or, indeed, an IaaS provider, or in some cases, people even make this claim about going with someone else's Data Center) - I always respond: "And why do you believe that you would do any better?" Every environment I've been involved in as an operations professional for the last 15 years has experienced downtime…

Lets hope they don't lose any of my data.

Re: Amazon Web Services are down

#126

Post by olegp »

ghshephard wrote: post #115
olegp wrote: post #101

Assuming the problem is indeed with EBS, I would say this should be a warning sign to anyone considering going with a PaaS provider, which Amazon is quickly becoming, instead of an IaaS provider like Slicehost or Linode. The increased complexity of their offering makes it more likely that things will break, leaving you locked in. I did a 15 minute talk on the subject, which you can check out here: http://iforum.com.u…

Every time someone makes the claim that downtime should be a warning sign about going with a PaaS provider (or, indeed, an IaaS provider, or in some cases, people even make this claim about going with someone else's Data Center) - I always respond: "And why do you believe that you would do any better?" Every environment I've been involved in as an operations professional for the last 15 years has experienced downtime…

The problem with Amazon is that despite touting an open API, their infrastructure internals and practices are a trade secret, so the likes of Eucalyptus are having to play catchup. In other words, I cannot replicate their infrastructure in my own data center, even I had the money to pay them. I suspect this is the main reason that Heroku didn't move off Amazon, and not the fact that Amazon was providing them great value for money.

There is definitely value in platforms, but those platforms should be built on completely open standards with no vendor lock in or influence, as is the case with OpenStack for example: http://www.theregister.co.uk/2011/02/10/rackspace_buys_opens...

If you didn't watch my talk, I should point out that I'm working on Akshell (http://www.akshell.com), which is itself a platform provider, so I am in fact agreeing with you.

Re: Amazon Web Services are down

#127

Post by stef25 »

jedberg wrote: post #15
kylec wrote: post #5

I guess this is one Reddit outage that can't be blamed on poor scaling

Thankfully, no. :) But yeah, right now we're shutting everything down to try and avoid possible data corruption. Once they restore service, hopefully we'll be able to come back quickly.

Amazon is really not being kind to you guys; I sort of hope you'll find an alternative solution fast!

Re: Amazon Web Services are down

#128

Post by billswift »

ghshephard wrote: post #115
olegp wrote: post #101

Assuming the problem is indeed with EBS, I would say this should be a warning sign to anyone considering going with a PaaS provider, which Amazon is quickly becoming, instead of an IaaS provider like Slicehost or Linode. The increased complexity of their offering makes it more likely that things will break, leaving you locked in. I did a 15 minute talk on the subject, which you can check out here: http://iforum.com.u…

Every time someone makes the claim that downtime should be a warning sign about going with a PaaS provider (or, indeed, an IaaS provider, or in some cases, people even make this claim about going with someone else's Data Center) - I always respond: "And why do you believe that you would do any better?" Every environment I've been involved in as an operations professional for the last 15 years has experienced downtime…

Maybe that's true for an individual, but from a broader (societal/economic) view this is a bigger problem, because it affects many more people.

Re: Amazon Web Services are down

#129

Post by asymptotic »

Amazon's EC2 SLA is extremely clear - a given region has an availability of 99.95%. If you're running a website and you haven't deployed across across more than one region then, by definition, your website will have 99.95% availailbility. If you want a higher level of availability use more than one region.

Amazon's EBS SLA is less clear, but they state that they expect an annual failure rate of 0.1-0.5%, compared to commodity hard-drive failure rates of 4%. Hence, if you wanted a higher level of data availability you'd use more than one EBS volume in different regions.

These outages are affecting North America, and not Europe and Asia Pacific. That's it. Why is this even news? Were you expecting 100% availability?

Post reply on HN