Live data from Hacker News

AWS us-east-1 down

news.ycombinator.com

51–60 of 331 posts

Re: AWS us-east-1 down

#51
post #4

I wonder if this is a coincidence or if us-east-1 is simply down enough that I'm just experiencing selection bias; but I posted a poll on twitter earlier today: https://twitter.com/dijit/status/1668678588713824257 Contents: > Has anyone ever actually had customers accept an outage because AWS was down; or is this just cloud evangelicalism copium? > [ ] Yeah, outages free pass > [ ] No, they say to use AZ's

I can think of more times where a whole AZ has had issues than times where just one AZ went dark and failover happened seamlessly.

Re: AWS us-east-1 down

#52
post #47

For those wondering: Currently PDT is 7 hours behind UTC. AWS can do so many things, reporting critical outage updates in UTC is not one of those things.

The inconsistency with timezones across different services in the AWS console has always baffled and annoyed me. Some places have a time without a timezone and I can never tell right away if it's utc, local time, or region time.

[deleted]

Re: AWS us-east-1 down

#53

Why is it always us-east-1 though? I have always stayed away from that region because it seems significantly less reliable than other regions.

us-east-1 is the largest region, so it is where changes meet scale.

It is also a massively complex beast in itself spanning dozens of datacenters with massive amounts of fiber between them. Much more fragile than having everything in a single building and as you scale up the number of components you increase the rate of failure.

Re: AWS us-east-1 down

#55

https://health.aws.amazon.com/health/status reports: Increased Error Rates and Latencies Jun 13 12:08 PM PDT We are investigating increased error rates and latencies in the US-EAST-1 Region. They list Lambda as the only affected service

status page won't load for me. are they still hosting their status page on their own infrastructure?

Re: AWS us-east-1 down

#56
post #26

us-east-1 seems to be very 'special' compared to the other regions - I wonder if they will ever align it with the rest of them.

Yep, it has issues so frequently. I wonder how many companies/teams start using AWS and blindly choose us-east-1 without realizing what they're getting into. It's also quite annoying sometimes that some things _need_ to be in us-east-1, and if e.g. you are using Terraform and specify a different default region, AWS will happily let you create useless resources in regions that aren't us-east-1 that then mysteriously b…

ACM certificates themselves can be had in any region (and you can use them for stuff like ELBs), but since the Cloudfront control plane is in us-east-1, if you want Cloudfront (and IIRC, also if you want custom domain names for an S3 bucket, but don't quote me on that) you'll have to create an additional certificate in us-east-1.

Sigh.

Re: AWS us-east-1 down

#57

Why is it always us-east-1 though? I have always stayed away from that region because it seems significantly less reliable than other regions.

I thought I read that this is where they deploy new changes first. Can anyone confirm?

When i worked at aws, IIRC, us-east-1 was one of the last regions we deployed to. So this is very confusing to me

Re: AWS us-east-1 down

#59
post #18

Earlier quoted context omitted.

AZs don't really help when it's AWS' own services across the entire region that break. Anecdotally, we have had customers accept outages that were out of our control without penalty.

Wild, that wouldn't have flown with datacenter providers having issues for my previous companies. AWS really does have an easier time than old school datacenter providers. I guess the complexity is higher but it's shocking that they can charge so much yet we hold them to a lower standard.

DCs are pretty static and offer way fewer services than AWS or any other public cloud.

I worked for one for some time and whenever we had issues, some people would call and ask if we were going bankrupt. It gave me a feeling they also have way smaller customers that might not understand the underlying stack.

Re: AWS us-east-1 down

#60

For those wondering: Currently PDT is 7 hours behind UTC. AWS can do so many things, reporting critical outage updates in UTC is not one of those things.

Semi-related: if you ever feel the need to report times to a global audience, not only make sure to always report the timezone (even if it is the same as the user's), but also use UTC offsets rather than timezone names.

Life is too short to remember what each timezone name means and converting to it, UTC offsets are much easier on the mental calculator.

Post reply on HN