Live data from Hacker News

Amazon EC2 currently down. Affecting Heroku, Reddit, Others

status.aws.amazon.com

271–280 of 311 posts

Re: Amazon EC2 currently down. Affecting Heroku, Reddit, Others

#271

Not wanting to shill for Google, but we've been very happy with Appengine's reliability - particularly since moving to the HR datastore. Have other Appengine users had any significant downtime apart from MS datastore issues?

What site are you running on App Engine?

Re: Amazon EC2 currently down. Affecting Heroku, Reddit, Others

#273
post #266

Earlier quoted context omitted.

Funny thing is, the last couple interviews I've had in Chicago and Silicon Valley, I actually get points when explaining caution is necessary when using AWS for production. A magic bullet it isn't.

I can't feel that Amazon are a bit of a Cassandra (mythological not the software) when these outages occur. They recommend that people failover to other availability zones but no one puts any effort into doing it then they get annoyed when a datacenter goes offline. Its not Amazons fault that you didn't make your service failure tolerant - its your fault!

perhaps they should make it easier to do so, as in having some default option you could select, at a premium of course.

Re: Amazon EC2 currently down. Affecting Heroku, Reddit, Others

#275
post #266

Earlier quoted context omitted.

Funny thing is, the last couple interviews I've had in Chicago and Silicon Valley, I actually get points when explaining caution is necessary when using AWS for production. A magic bullet it isn't.

I can't feel that Amazon are a bit of a Cassandra (mythological not the software) when these outages occur. They recommend that people failover to other availability zones but no one puts any effort into doing it then they get annoyed when a datacenter goes offline. Its not Amazons fault that you didn't make your service failure tolerant - its your fault!

Just like: passwords, backups, etc.

It reminds me of my father: I used to interrupt him with "I know dad!" when he was chastising me. His response was simple "If you knew, then why did[n't] you do it?"

Re: Amazon EC2 currently down. Affecting Heroku, Reddit, Others

#276

Earlier quoted context omitted.

Funny thing is, the last couple interviews I've had in Chicago and Silicon Valley, I actually get points when explaining caution is necessary when using AWS for production. A magic bullet it isn't.

Whats the alternative? Building your own is certainly not.

Did you know that many websites operated BEFORE Amazon Web Services existed? Perhaps going back to 2008 could give us some ideas for alternate deployment methodologies...

Re: Amazon EC2 currently down. Affecting Heroku, Reddit, Others

#277
post #266

Earlier quoted context omitted.

Funny thing is, the last couple interviews I've had in Chicago and Silicon Valley, I actually get points when explaining caution is necessary when using AWS for production. A magic bullet it isn't.

I can't feel that Amazon are a bit of a Cassandra (mythological not the software) when these outages occur. They recommend that people failover to other availability zones but no one puts any effort into doing it then they get annoyed when a datacenter goes offline. Its not Amazons fault that you didn't make your service failure tolerant - its your fault!

My question is this...why doesn't a service like Heroku which acts as a Paas...have this built in? I'm on their blog now trying to understand their complete setup..

Re: Amazon EC2 currently down. Affecting Heroku, Reddit, Others

#279
post #266

Earlier quoted context omitted.

Funny thing is, the last couple interviews I've had in Chicago and Silicon Valley, I actually get points when explaining caution is necessary when using AWS for production. A magic bullet it isn't.

I can't feel that Amazon are a bit of a Cassandra (mythological not the software) when these outages occur. They recommend that people failover to other availability zones but no one puts any effort into doing it then they get annoyed when a datacenter goes offline. Its not Amazons fault that you didn't make your service failure tolerant - its your fault!

I'm seeing a lot of these type of comment. The thing is, AWS completely crapped out. Don't believe their status updates that make it sound like it was a tiny little area of their data center. It was pretty much the entire zone and then whenever there is an outage affecting an entire zone it brings down global services and even other zones as well.

We had servers in the bad zone and started having load issues. When I went to use the cool cloud features that are made for this, the entire thing completely fell on its face. I couldn't launch new EC2 servers either because the API was so bogged down, or the new zone I was launching in was restricted because of load.

Basically, the thing that nobody keeps in mind when they think it's so cool that you can spin up servers to work around outages is that EVERYONE IS DOING THAT. This is Amazon's entire selling point and when it comes to doing it, it doesn't work!

We were lucky to get some new servers launched before the API pretty much completely went down. They started giving everyone errors saying request limit exceeded. The forums were full of people asking about it.

ELB, Elastic IP, and other services not associated with a single availability zone completely failed. I keep seeing comments saying that if people designed their stuff right, they wouldn't have an issue. That's just completely bull, AWS has serious design flaws and they show up at every outage. It's NOT just people relying on a single zone.

Re: Amazon EC2 currently down. Affecting Heroku, Reddit, Others

#280

Earlier quoted context omitted.

He knows you were joking. It's a bad joke. "It's a joke" is not a magic bullet that means you can do no wrong. Poisonous ideas spread as jokes. That is one of the ways they spread. A person thinking well about the issue wouldn't find the joke funny because it doesn't make sense. The joke relies on some poisonous, bad thinking to be understood. It has bad assumptions, and a bad way of looking at the world, built in.

Let me explain to you why it is a funny joke. It is funny because it involves Amazon undertaking massive technical measures, with huge reputational damage in order to try to kill a story which is primarily not spreading via Amazon-hosted sites anyway. It's akin to a man with athlete's foot deciding to remedy it by discharging a shotgun into his leg.

It's easy to understand shooting a leg with a shotgun. That's a simple thing.

The Amazon thing in question is far more complicated, and far harder to understand.

Thinking they are "akin" is a mistake. It shows you're thinking about it wrong and failing to recognize how completely different they are.

One isn't going to confuse anyone or be misunderstood, the other will confuse most of the population and be misunderstood by most people.

One, if someone misunderstood, only involves one individual being an idiot. The other involves a large company being evil and thus can help feed conspiracy theories.

I'm not sure if you are aware of the difficulty of Amazon doing this. Suppose Jeff Bezos wants to do it. He can't simply order people to do it because they will refuse and leak it to the media and he'll look really bad and then he'll definitely have to make sure to try super hard for there to be no outages anytime soon.

Shooting yourself in the foot is stupid but easy. Doing this is stupid and essentially impossible. To think it's possible requires thinking that Amazon has a culture of unthinking obedience, or has an evil culture that all new hires are told about and don't leak to the media. Totally different.

Casually talking about impossible, evil conspiracies by big business, as if they are even possible, is a serious slander against those businesses, capitalism, and logic. Slandering a bunch of really good things -- especially ones that approximately 50% of US voters want to harm -- and then saying "it's just a joke, it's funny" is bad.

Post reply on HN