Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is Northern Virginia still the least reliable AWS region?

statusgator.com

Post reply on HN
Print view View original

71–80 of 84 posts

Re: Is Northern Virginia still the least reliable AWS region?

#71

Post by g947o »

yibers wrote: post #3

Ass covering-wise, you are probably better off going down with everyone else on us-east-1. The not so fun alternative: being targeted during an RCA explaining why you chose some random zone no one ever heard of.

> explaining why you chose some random zone no one ever heard of

Is this from real experience of something that actually happened, or just imagined?

The only things that matter in a decision are:

* Services that are available in the region

* (if relevant and critical) Latency to other services

* SLAs for the region

Everything else is irrelevant.

If you think AWS is so bad that their SLAs are not trustworthy, that's a different problem to solve.

Re: Is Northern Virginia still the least reliable AWS region?

#72

Post by redditor98654 »

Fhch6HQ wrote: post #42
jordanb wrote: post #38

Earlier quoted context omitted.

Istr major resource unavailability in US-East-2 during one of the big US-East-1 outages because people were trying to fail over. Then a week later there was a US-East-2 outage that didn't make the news. So if you tried to be "smart" and set up in Ohio you got crushed by the thundering herd coming out of Virginia and then bit again because aws barely cares about you region and neither does anyone else. The truth is Am…

This is an interesting point. As recently as mid-2023 us-east-2 was 3 campuses with a 5 building design capacity at each. I know they've expanded by multiples since, but us-east-1 would still dwarf them. Makes one wonder, does us-west-2 have the capacity to take on this surge?

us-west-2 is indeed very large, but will still not be able to take a full failover from us-east-1

Re: Is Northern Virginia still the least reliable AWS region?

#73

Post by htrp »

Manouchehri wrote: post #53
kankerlijer wrote: post #25

There are only two kinds of cloud regions: the ones people complain about and the ones nobody uses

Yeah, I was often the single source of reporting Claude outages (or even missing support completely) on less commonly used Amazon Bedrock regions.

Which regions were you using ? ( Thought claude had global inference support that routed to all regions)

Re: Is Northern Virginia still the least reliable AWS region?

#74

Post by [deleted] »

TonyCoffman wrote: post #68
paulddraper wrote: post #58

Earlier quoted context omitted.

IAM and Route53 have dependencies on us-east-1. AWS Organizations/Account management is us-east-1. And if you want a CDN with a custom hostname and want TLS…you have to use us-east-1.

The Route53 control plane is in us-east-1, with an optional temporary auto-failover to us-west-2 during outages. The data plane for public zones is globally distributed and highly resilient, with a 100% SLA. It continues to serve DNS records during regular control plane outages in us-east-1, but access to make changes is lost during outages. CloudFront CDN has a similar setup. The SSL certificate and key have to be h…

[deleted]

Re: Is Northern Virginia still the least reliable AWS region?

#75

Post by throwawaysleep »

Esophagus4 wrote: post #70
nemomarx wrote: post #35

Earlier quoted context omitted.

It's about risk profile. The question isn't "which region goes down the least" but "how often will I be blamed for an outage." If you never get blamed for a US east outage, that's better than us-east-2 if that could get you blamed 0.5% of the time when it goes down and us1 isn't down or etc

But ise1 is down 4x more than use2 (AWS closely guards the numbers and won’t release them, but that is what I’ve seen from 3rd party analysis). Don’t you want your customers to say, “wow, half the internet was down today but XYZ service was up with no issues! I love them.” I can’t tell if it’s you thinking this way, or if your company is setup to incentivize this. But either way, I think it’s suboptimal. That’s not a…

> Don’t you want your customers to say, “wow, half the internet was down today but XYZ service was up with no issues! I love them.”

See any companies getting credit for it in the last AWS outage? I didn't. My employers didn't reward vendors who stayed up during it.

Re: Is Northern Virginia still the least reliable AWS region?

#76

Post by throwawaysleep »

TheNewsIsHere wrote: post #39
Esophagus4 wrote: post #34

Earlier quoted context omitted.

Bizarre way of making decisions. us-east-2 is objectively a better region to pick if you want US east, yet you feel safer picking use1 because “I’m safer making a worse decision that everyone understands is worse, as long as everyone else does it as well.”

I also don’t understand this. US-East-2 staying up isn’t my responsibility. If I need my own failover, I’m going to select a different region anyway. And it’s not like US-East-2 isn’t already huge and growing. It’s effectively becoming another US-East-1.

> US-East-2 staying up isn’t my responsibility.

No, but you can be blamed if other things are up and yours is not. If everyone's stuff is down, it is just a natural disaster.

Re: Is Northern Virginia still the least reliable AWS region?

#77

Post by Esophagus4 »

throwawaysleep wrote: post #75
Esophagus4 wrote: post #70

Earlier quoted context omitted.

But ise1 is down 4x more than use2 (AWS closely guards the numbers and won’t release them, but that is what I’ve seen from 3rd party analysis). Don’t you want your customers to say, “wow, half the internet was down today but XYZ service was up with no issues! I love them.” I can’t tell if it’s you thinking this way, or if your company is setup to incentivize this. But either way, I think it’s suboptimal. That’s not a…

> Don’t you want your customers to say, “wow, half the internet was down today but XYZ service was up with no issues! I love them.” See any companies getting credit for it in the last AWS outage? I didn't. My employers didn't reward vendors who stayed up during it.

We got credit for it.

Shame about your employer, though.

Re: Is Northern Virginia still the least reliable AWS region?

#79

Post by nineteen999 »

bob1029 wrote: post #59

I think if you need something more reliable than us-east-1 that you should be hosting on prem in facilities you own and operate. There aren't that many businesses that truly can't handle the worst case (so far) AWS outage. Payment processing is the strongest example I can come up with that is incompatible with the SLA that a typical cloud provider can offer. Visa going down globally for even a few minutes might be wo…

> It's a hell of a lot easier to just go down with everyone else, apologize on Twitter, and enjoy a forced snow day.

You forget things like emergency services. If we were to rely on AWS (even with a backup/DR zone in another region), and were to go down with everyone else and twiddle our fingers, houses burn down, people die, and our company has to pay abatements to the govt.

Re: Is Northern Virginia still the least reliable AWS region?

#80

Post by roger01 »

jl6 wrote: post #61
bob1029 wrote: post #59

I think if you need something more reliable than us-east-1 that you should be hosting on prem in facilities you own and operate. There aren't that many businesses that truly can't handle the worst case (so far) AWS outage. Payment processing is the strongest example I can come up with that is incompatible with the SLA that a typical cloud provider can offer. Visa going down globally for even a few minutes might be wo…

> worst case (so far) It’s kind of amazing that after nearly 20 years of “cloud”, the worst case so far still hasn’t been all that bad. Outages are the mildest type of incident. A true cloud disaster would be something like a major S3 data loss event, or a compromise of the IAM control plane. That’s what it would take for people to take multi-region/multi-cloud seriously.

> compromise of the IAM control plane

You mean like stealing the master keys for Azure? Oh wait a minute...

Post reply on HN