251–260 of 1001 posts
Are the actual services down, or is it just the console and/or login page? For example, the sign-up page appears to be working: https://portal.aws.amazon.com/billing/signup#/start Are websites that run on AWS us-east up? Are the AWS CLIs working?
I can still hit EC2 boxes and networking is okay. DynamoDB is 100% down for the count, every request is an Internal Server Error.
At this point I have no idea why anyone would put anything in us-east-1. Also isolation is not as good as they would have you believe: I am unable to login to AWS Quicksight in us-west-2...
I love that every time this happens, 100% of the services on https://status.aws.amazon.com are green.
Mom: Alexa, did you break something?
Alexa: No.
M: Really? What's this? 500 Internal server error
A: ok maybe management console is down
M: Anything else?
A: ...
A: ... ok maybe cloudwatch logs
M: Ah hah. What else?
A: That's it, I swear!
M: 503 ClientError
A: ...well okay secretsmanager might be busted too...
I'm now getting failures searching for products on Amazon.com itself. This is somewhat surprising, as the narrative always was that Amazon didn't do a great job of dogfooding their own cloud platform.
It's funny that the first place I go to learn about the outage is Hacker News and not https://status.aws.amazon.com/ (it's still reports everything to be "operating normally"...)
As of right now, this thread and one update from a twitter user, https://twitter.com/SiteRelEnby/status/1468253604876333059 are all we have. I went into disaster recovery mode when I saw our traffic dropped to 0 suddenly at 10:30am ET. That was just the SQS/something else preventing our ELB logs from being extracted to DataDog though.
The Amazon.com storefront was giving me issues loading search results — this is the worst possible time of year for Amazon to have issues. It’s horrifying and awesome to imagine hundreds of thousands (if not millions) of dollars of lost orders an hour — just from sluggish load times. Hugops to those dealing with this.
Looks like they've acknowledged it on the status page now. https://status.aws.amazon.com/ > 8:22 AM PST We are investigating increased error rates for the AWS Management Console. > 8:26 AM PST We are experiencing API and console issues in the US-EAST-1 Region. We have identified root cause and we are actively working towards recovery. This issue is affecting the global console landing page, which is also hosted in US…
A former colleague told me years ago that us-east-1 is basically the guinea pig where changes get tested before being rolled out to the other regions, and as a result is less stable than the others. Does anyone know if there's any truth to this?