Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS us-east-2 outage

news.ycombinator.com

Post reply on HN
Print view View original

131–140 of 254 posts

Re: AWS us-east-2 outage

#131

Post by blahyawnblah »

silverlyra wrote: post #33
MikeMacMan wrote: post #15

I'm seeing issues in 2a but not 2b. Anyone having issues in 2b?

AWS availability zones are randomly shuffled for each AWS account – your us-east-2a won't (necessarily) be the same as another user's (or even another account in the same organization): https://docs.aws.amazon.com/ram/latest/userguide/working-wit... You'll need to see which availability zone ID (e.g., use2-az3) corresponds to each zone in your account: https://aws.amazon.com/premiumsupport/knowledge-center/vpc-m... e…

I wonder if this is done because people have a tendency or something to always create resources in 'A' (or some other AZ) and this helps spread things around.

And if I would have read the page the link points to better, that's exactly the reason

Re: AWS us-east-2 outage

#132

Post by boomer918 »

dangero wrote: post #101
poxrud wrote: post #97

The fact that so many popular sites/services are experiencing issues due to a single AZ failure makes me think that there is a serious shortage of good cloud architects/engineers in the industry. It would be one thing if this was a Regional failure, but a single AZ failure should not have any noticeable effect.

For most businesses a little down time here and there is a calculated risk versus more complex infrastructure. You can’t assume all the cloud architects are idiots — they have to report their task list and cost of infrastructure to someone who can give feedback on various options based on comparative resource requirements and risks. Zone downtime still falls under an AWS SLA so you know about how much downtime to acc…

A multi-az deployment is a checkbox in most AWS services, e.g. ASGs, RDS, load balancers, etc. Someone didn't check that box because they didn't know about it, there isn't much complexity in it.

Re: AWS us-east-2 outage

#133

Post by trelliscoded »

poxrud wrote: post #97

The fact that so many popular sites/services are experiencing issues due to a single AZ failure makes me think that there is a serious shortage of good cloud architects/engineers in the industry. It would be one thing if this was a Regional failure, but a single AZ failure should not have any noticeable effect.

Replicating a huge database between AZs, let alone regions, can be an enormously expensive ongoing operational cost. Not everyone can afford it.

Re: AWS us-east-2 outage

#134

Post by schroeding »

boomer918 wrote: post #132
dangero wrote: post #101

Earlier quoted context omitted.

For most businesses a little down time here and there is a calculated risk versus more complex infrastructure. You can’t assume all the cloud architects are idiots — they have to report their task list and cost of infrastructure to someone who can give feedback on various options based on comparative resource requirements and risks. Zone downtime still falls under an AWS SLA so you know about how much downtime to acc…

A multi-az deployment is a checkbox in most AWS services, e.g. ASGs, RDS, load balancers, etc. Someone didn't check that box because they didn't know about it, there isn't much complexity in it.

Aren't multi-az deployments more expensive? That would be a valid reason not to check this checkbox, if your business can survive a bit of downtime here and there.

Re: AWS us-east-2 outage

#135

Post by kbumsik »

poxrud wrote: post #97

The fact that so many popular sites/services are experiencing issues due to a single AZ failure makes me think that there is a serious shortage of good cloud architects/engineers in the industry. It would be one thing if this was a Regional failure, but a single AZ failure should not have any noticeable effect.

Multi-AZ architecture just double the cost at least, and it tends to cost even much more if the business is small. Good engineers find the balance between the cost and the availability.

Re: AWS us-east-2 outage

#137

Post by phamilton »

schroeding wrote: post #134
boomer918 wrote: post #132

Earlier quoted context omitted.

A multi-az deployment is a checkbox in most AWS services, e.g. ASGs, RDS, load balancers, etc. Someone didn't check that box because they didn't know about it, there isn't much complexity in it.

Aren't multi-az deployments more expensive? That would be a valid reason not to check this checkbox, if your business can survive a bit of downtime here and there.

Most of that expense is just the cost of a hot failover, but there is some additional cost around inter-AZ data transfer. If someone is not checking the boxes for cost reasons, I would be surprised if they had failovers in the same AZ. It seems more likely they just don't have failovers.

Re: AWS us-east-2 outage

#139

Post by harryh »

rjh29 wrote: post #105
poxrud wrote: post #97

The fact that so many popular sites/services are experiencing issues due to a single AZ failure makes me think that there is a serious shortage of good cloud architects/engineers in the industry. It would be one thing if this was a Regional failure, but a single AZ failure should not have any noticeable effect.

It's a game theory thing. If everyone stays single AZ, everyone goes down at the same time so nobody gets blamed. Somehow the blame falls on AWS instead!

I think you're confusing availability zones with regions in this comment.

AWS AZs don't even have consistent naming across AWS accounts.

Re: AWS us-east-2 outage

#140

Post by res0nat0r »

brasic wrote: post #86
packetslave wrote: post #45

Update from AWS: they lost power to (part of?) a single DC in the use2-az1 availability zone. 10:25 AM PDT We can confirm that some instances within a single Availability Zone (USE2-AZ1) in the US-EAST-2 Region have experienced a loss of power. The loss of power is affecting part of a single data center within the affected Availability Zone. Power has been restored to the affected facility and at this stage the major…

I thought that AWS availability zones were intentionally not canonically named to prevent everyone from adding stuff to AZ “A”. So my us-east-1 zone “A” might be your “B”. But that system breaks down here when you need to know whether you are in an affected zone. Is there a way to map an account’s AZ name to the canonical one which apparently exists?

This is true, at the AWS account level. us-east-2a for my account may map to the internal use2-az1, but in your account us-east-2a may map internally to use2-az2.
Post reply on HN