https://health.aws.amazon.com/health/status now it is showing. Lot of services are impacted.
AWS us-east-1 down
151–160 of 331 posts
Re: AWS us-east-1 down
#152Earlier quoted context omitted.
Semi-related: if you ever feel the need to report times to a global audience, not only make sure to always report the timezone (even if it is the same as the user's), but also use UTC offsets rather than timezone names. Life is too short to remember what each timezone name means and converting to it, UTC offsets are much easier on the mental calculator.
just report it in epoch seconds
Or stardate ¯\_(ツ)_/¯
Re: AWS us-east-1 down
#153I wonder if this is a coincidence or if us-east-1 is simply down enough that I'm just experiencing selection bias; but I posted a poll on twitter earlier today: https://twitter.com/dijit/status/1668678588713824257 Contents: > Has anyone ever actually had customers accept an outage because AWS was down; or is this just cloud evangelicalism copium? > [ ] Yeah, outages free pass > [ ] No, they say to use AZ's
I can think of more times where a whole AZ has had issues than times where just one AZ went dark and failover happened seamlessly.
s/whole AZ/whole region/
Re: AWS us-east-1 down
#154Earlier quoted context omitted.
No AWS region is in a single building, they aren't amateurs like Azure. Each region is at least 3 AZs, which is at least one physical DC.
And yet it's AWS that's down.
Touché. Still I'd rate the overall reliability of AWS higher than Azure; and even if that weren't the case, security issues make Azure look like a very poor choice.
Re: AWS us-east-1 down
#155I love how their status says that services are just degraded
Re: AWS us-east-1 down
#156Earlier quoted context omitted.
It's also usually extremely US-centric. Nobody outside of North America has any idea what "PDT" or "Mountain Time" means.
3 years ago, when I started work for my current employer, I noticed in Slack that everyone was reckoning time in "Standard Time" year-round. Now imagine my chagrin because I live in Arizona, and "Mountain Standard Time" does not change for DST. Therefore, all my coworkers were citing nonsensical, nonexistent time zones and it was messing up my ability to convert back and forth. Come to find out that this was some sor…
How does this generate off-by-one errors? I am also part of a company with employees in pretty much every timezone, but when they create a meeting the meeting invitation is programmed with the correct timezone so in my Calendar it always shows what time the meeting is going to be for me. I never even have to think what timezone the organizer is...
Re: AWS us-east-1 down
#157It doesn't matter if your infra is in another region, because there will almost always be transitive dependencies on us-east-1. IAM is deployed in us-east-1 and there will always be a transitive dependency on us-east-1
Control plane will almost always be impacted, I agree.
Our data plane was fine (for example, ec2 instances and s3 buckets in other regions were fine).
Re: AWS us-east-1 down
#158It's fun watching each service fail sequentially while the aws service dashboard just updates them to "Informational" status, whatever that means. Even management console is down, and their suggested region specific workaround does not work, at least for us-east-1. I can see some processes via api but I don't have code prepared for monitoring every service from my local.
And now the service health page is down.
Re: AWS us-east-1 down
#159Toast POS is down 100%, don't go out to lunch.
I can see some restaurants just comp’ing the tickets out and having toast foot the bill in lost sales
You say that as if it's as easy as sending Toast the bill and Toast just going "Yeah okay we'll pay".
When the POS system goes down, restaurants take down credit card numbers, and then charge them later when the POS comes back up.
Re: AWS us-east-1 down
#160I wonder if this is a coincidence or if us-east-1 is simply down enough that I'm just experiencing selection bias; but I posted a poll on twitter earlier today: https://twitter.com/dijit/status/1668678588713824257 Contents: > Has anyone ever actually had customers accept an outage because AWS was down; or is this just cloud evangelicalism copium? > [ ] Yeah, outages free pass > [ ] No, they say to use AZ's
it's important to inform customers about the resiliency of their systems and let them pick how far they are going to invest for it. then you get to eat popcorn when stuff explodes. * single server event. $ * multi server event. $$ * single az event. $$$ * multi az event. $$$$ * global provider event. $$$$$ * cross provider event. $$$$$$ * alien invasion. $$$$$$$$$$$$$$
Always be prepared for alien invasion