Live data from Hacker News

Amazon EC2 currently down. Affecting Heroku, Reddit, Others

status.aws.amazon.com

161–170 of 311 posts

Re: Amazon EC2 currently down. Affecting Heroku, Reddit, Others

#161

Earlier quoted context omitted.

Amazon is downplaying the problem. It is affecting many large sites.

Many large sites which aren't properly architecting their infrastructure to deal with the commodity nature of the cloud.

Including companies like Amazon that aren't properly architecting their control plane.

Re: Amazon EC2 currently down. Affecting Heroku, Reddit, Others

#162

Earlier quoted context omitted.

To be fair, AWS downtime always make the news because they affect a lot of majors websites, but that doesn't mean an average sysadmin (or devops, whatever) would do better in term of uptime with his own bay and his toys.

But this is part of the problem: we have multiple web properties, and the fact that AWS issues can affect all of them at once is a huge downside. Certainly, if we ran on metal, we would have hardware fail, but failures would be likely to be better-isolated than at Amazon.

Ah the argument of iron over cloud. It would cose significantly more to buy the hardware to be redundant enough to avoid a situation like today. If these large sites still aren't buying their own hardware that seems go along with that thinking.

Re: Amazon EC2 currently down. Affecting Heroku, Reddit, Others

#163
post #51

Earlier quoted context omitted.

Realistically, it's at least in part because everyone defaults to the East region. So it's the most crowded and demanding on the system.

Yeah, it's got to be much larger than the other regions, so it makes sense that we see more errors. Since error_rate = machines * error_rate_per_machine.

The whole region is down, you just calculated the chance of at least one machine having an error.

Re: Amazon EC2 currently down. Affecting Heroku, Reddit, Others

#164
post #4

Per the linked dashboard, some instances in a single AZ in a single Region are having storage issues. Calling EC2 "down" is a bit dramatic, provided AMZN are being sincere with their status reports. Any system that can competently fail over to another AZ will be unaffected.

I would agree with you, but Amazon is just downright dishonest in their reports, which makes me sad, because I love Amazon. Go look at the past reports, they've never shown a red market, only "degraded performance" even when services for multiple availability zones went down at the same time due to their power outage (so had you architected to multiple AZs you were still fucked). When they have a single AZ go down, t…

They reserve the red mark for "The world just exploded". Orange is reserved for "The datacenter got bombed".

Re: Amazon EC2 currently down. Affecting Heroku, Reddit, Others

#165

Earlier quoted context omitted.

. . . or the problem is inherently complex.

Surely true, but that's the purpose of a system in the first place: to manage complexity and make it predictable. You could argue that we have such a system in place, given how well the Internet works overall. The fact that this system has problems goes against what I believe is fully evident proof that such a system can, in fact, work even better. We're not talking about a leap in order of magnitude of complexity he…

The internet works because it provides very weak consistency guarantees compared to what businesses might require out of an EC2 management console. (IMO.)

Re: Amazon EC2 currently down. Affecting Heroku, Reddit, Others

#166
post #126
post #31

The N. Virginia datacenter has been historically unreliable. I moved my personal projects to the West Coast (Oregon and N. California) and I have seen no significant issues in the past year. N. Virginia is both cheaper and closer to the center of mass of the developed world. I'm surprised Amazon hasn't managed to make it more reliable.

You can move your projects on demand?

[deleted]

Re: Amazon EC2 currently down. Affecting Heroku, Reddit, Others

#167
post #156

Maybe I'm being daft. Why does this sh*t never seem to affect www.amazon.com

Amazon doesn't run their site off the same set of EC2 servers and systems that everyone else does. They just used their experience to build it.

That would be the ultimate testament, maybe the only stunt they could pull to PR themselves out of this mess.. if www.amazon.com ran on the same set of EC2 servers and systems as us muggles
Post reply on HN