I love that every time this happens, 100% of the services on https://status.aws.amazon.com are green.
Well yea, it's eventually consistent ;)
AWS us-east-1 outage
401–410 of 1001 posts
Re: AWS us-east-1 outage
#402I worked at a company that hired an ex-Amazon engineer to work on some cloud projects. Whenever his projects went down, he fought tooth and nail against any suggestion to update the status page. When forced to update the status page, he'd follow up with an extremely long "post-mortem" document that was really just a long winded explanation about why the outage was someone else's fault. He later explained that in his…
being at fault for an outage was one of the worst things that could happen to you
Imagine how stressful life would be thinking that you had to be perfect all the time.
Re: AWS us-east-1 outage
#403us-east-1 is so unreliable that it probably should be nuked. It's the region with the worst track record. I guess it doesn't help that is one of the oldest.
Re: AWS us-east-1 outage
#404AWS Connect is down, so our customer support phone system is down with it
Highly recommend talking to your account team to recommend regional failovers and DR for Amazon Connect! With enough feedback from customers, stuff like this can get prioritized.
Thanks, will definitely do that!
Re: AWS us-east-1 outage
#405noob question: Aren't companies using several regions for availability and redundancy?
Re: AWS us-east-1 outage
#406So, we're getting failures (for customers) trying to use amazon pay from our site. AFAIK there is no "status page" for Amazon Pay, but the rest of Amazon's services seem to be a giant Rube Goldberg machine so it's hard to imagine this isn't too.
Re: AWS us-east-1 outage
#407Re: AWS us-east-1 outage
#408Earlier quoted context omitted.
Would they? Having 3 outages in a year sounds like an organization problem. Not enough safeguards to prevent very routine human errors. But instead of worrying about that we just assign a guy to take the fall
Well if John caused 3 outages and and his peers Sally and Mike each caused 0, it's worth taking a deeper look. There's a real possibility he's getting screwed by a messed up org, also he could be doing slapdash work or he seriously might not undertsand the seriousness of an outage.
Or John's work is in frontline production use and Sally's and Mike's is not, so there's different exposure.
Re: AWS us-east-1 outage
#409Funny, I just asked Alexa to set a timer and she said there was a problem doing that. Apparently timers require functioning us-east-1 now.
I can’t turn on my lights… the future is weird
Re: AWS us-east-1 outage
#410Earlier quoted context omitted.
If you count any time AWS is having a problem that impacts our production workloads then I think it's about 5:1. Dealing with "AWS is down" outages are easy because I can just sit back and grab some popcorn, it's the "dammit I know this is AWS's fault" outages that are a PITA because you count yourself lucky to even get a report in your personalized dashboard.
Yep. Random aside: any chance you are related to the Calculus on Manifolds Spivak?
I had to log in to say, that one of my favorite quotes of all time I found in Calculus on Manifolds.
He says that any good theorem is worth generalizing, and I've generalized that to any life rule.