Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS Glue/Athena/Redshift outage

status.aws.amazon.com

Post reply on HN
Print view View original

11–17 of 17 posts

Re: AWS Glue/Athena/Redshift outage

#11

Post by Johnny555 »

thinkingkong wrote: post #5
nnx wrote: post #4

Should edit title to add “in us-east-1”. Other regions are unimpacted.

There are so many outages in us-east-1. I've heard the reason is because that's where they roll out maintenance first or something along those lines. Just look at this list of outages on Wikipedia [1] and scan for US-east-1, North Virginia, or "Northeast" (all the same places). Just don't use US-EAST-1 as your region. 1. https://en.wikipedia.org/wiki/Timeline_of_Amazon_Web_Service...

I've heard the reason is because that's where they roll out maintenance first

That doesn't make sense - why would they do maintenance in their largest (and oldest) region first? I'd expect them to roll out changes to smaller regions first so problems will affect fewer users.

I think the more likely explanation is that it's their largest (and oldest) region.

Re: AWS Glue/Athena/Redshift outage

#12

Post by swasheck »

Johnny555 wrote: post #11
thinkingkong wrote: post #5

Earlier quoted context omitted.

There are so many outages in us-east-1. I've heard the reason is because that's where they roll out maintenance first or something along those lines. Just look at this list of outages on Wikipedia [1] and scan for US-east-1, North Virginia, or "Northeast" (all the same places). Just don't use US-EAST-1 as your region. 1. https://en.wikipedia.org/wiki/Timeline_of_Amazon_Web_Service...

I've heard the reason is because that's where they roll out maintenance first That doesn't make sense - why would they do maintenance in their largest (and oldest) region first? I'd expect them to roll out changes to smaller regions first so problems will affect fewer users. I think the more likely explanation is that it's their largest (and oldest) region.

an aws tam once told me the same thing. us-east-1a gets the new stuff first. i never validated it against anything other than this one person's statement.

Re: AWS Glue/Athena/Redshift outage

#13

Post by discodave »

thinkingkong wrote: post #5
nnx wrote: post #4

Should edit title to add “in us-east-1”. Other regions are unimpacted.

There are so many outages in us-east-1. I've heard the reason is because that's where they roll out maintenance first or something along those lines. Just look at this list of outages on Wikipedia [1] and scan for US-east-1, North Virginia, or "Northeast" (all the same places). Just don't use US-EAST-1 as your region. 1. https://en.wikipedia.org/wiki/Timeline_of_Amazon_Web_Service...

It's the oldest region, which means:

* It's the largest region (ever had an unexpected scaling bug?).

* It has more legacy stuff lying around. For example, old regions have EC2 Classic, while new regions are VPC only.

* There are more customers there. More whales, more use cases.

Most AWS teams explicitly try not to deploy to us-east-1 first, but because us-east-1 is so different on so many dimensions, it is more likely to have issues that dont manifest elsewhere.

(Source: An AWS Engineer)

Re: AWS Glue/Athena/Redshift outage

#14

Post by ecnahc515 »

swasheck wrote: post #12
Johnny555 wrote: post #11

Earlier quoted context omitted.

I've heard the reason is because that's where they roll out maintenance first That doesn't make sense - why would they do maintenance in their largest (and oldest) region first? I'd expect them to roll out changes to smaller regions first so problems will affect fewer users. I think the more likely explanation is that it's their largest (and oldest) region.

an aws tam once told me the same thing. us-east-1a gets the new stuff first. i never validated it against anything other than this one person's statement.

"1a" in this context means nothing. The AZ assignments each account gets is random. us-east-1a is probably a different data center for you than me.

Re: AWS Glue/Athena/Redshift outage

#15

Post by [deleted] »

swasheck wrote: post #12
Johnny555 wrote: post #11

Earlier quoted context omitted.

I've heard the reason is because that's where they roll out maintenance first That doesn't make sense - why would they do maintenance in their largest (and oldest) region first? I'd expect them to roll out changes to smaller regions first so problems will affect fewer users. I think the more likely explanation is that it's their largest (and oldest) region.

an aws tam once told me the same thing. us-east-1a gets the new stuff first. i never validated it against anything other than this one person's statement.

[deleted]

Re: AWS Glue/Athena/Redshift outage

#16

Post by teej »

thinkingkong wrote: post #5
nnx wrote: post #4

Should edit title to add “in us-east-1”. Other regions are unimpacted.

There are so many outages in us-east-1. I've heard the reason is because that's where they roll out maintenance first or something along those lines. Just look at this list of outages on Wikipedia [1] and scan for US-east-1, North Virginia, or "Northeast" (all the same places). Just don't use US-EAST-1 as your region. 1. https://en.wikipedia.org/wiki/Timeline_of_Amazon_Web_Service...

Redshift changes roll out in us-east-1 after other regions though, so I imagine the root cause is something else.
Post reply on HN