Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS us-east-1 down

news.ycombinator.com

Post reply on HN
Print view View original

211–220 of 331 posts

Re: AWS us-east-1 down

#211

Post by dmckeon »

xyst wrote: post #112
kobalsky wrote: post #71

Earlier quoted context omitted.

it's important to inform customers about the resiliency of their systems and let them pick how far they are going to invest for it. then you get to eat popcorn when stuff explodes. * single server event. $ * multi server event. $$ * single az event. $$$ * multi az event. $$$$ * global provider event. $$$$$ * cross provider event. $$$$$$ * alien invasion. $$$$$$$$$$$$$$

Just need to deploy your service on Mars AND Earth. Duh

And you thought the current time zone confusion was bad. Now you have two sets of time zones, and a varying delay of about 5 to 21 minutes between them. Oh, the joy!

Re: AWS us-east-1 down

#213

Post by jedberg »

Matthias247 wrote: post #104
dijit wrote: post #46

Earlier quoted context omitted.

It's the: * Largest (DDoS'd most, most complex, scaling issues etc) * Oldest (More time for weird idiosyncrasies to take hold) * Where most testing happens * Where new products are deployed first

1) and 2) certainly apply. 3) and 4) don't. Testing in the largest region is one of the biggest anti-patterns.

4 is still generally true. Most new features drop in us-east-1 on launch day.

Re: AWS us-east-1 down

#214

Post by chrisco255 »

ulrashida wrote: post #203
throaway87c10f0 wrote: post #164

Mysterious lack of "AWS is bad for the internet because it is so centralized" dialog up in here. edit: for those that would downvote: HN _just_ yesterday: https://news.ycombinator.com/item?id=36295352 https://news.ycombinator.com/item?id=36295305

Why a throwaway for this post? Not like this is some deep whistleblowing or career risk.

Maybe they work for Amazon.

Re: AWS us-east-1 down

#215

Post by jmacjmac »

dijit wrote: post #4

I wonder if this is a coincidence or if us-east-1 is simply down enough that I'm just experiencing selection bias; but I posted a poll on twitter earlier today: https://twitter.com/dijit/status/1668678588713824257 Contents: > Has anyone ever actually had customers accept an outage because AWS was down; or is this just cloud evangelicalism copium? > [ ] Yeah, outages free pass > [ ] No, they say to use AZ's

Maybe cheaper regions have more users and have higher outage rates

Re: AWS us-east-1 down

#216

Post by jcims »

luhn wrote: post #204
adubashi wrote: post #118

It doesn't matter if your infra is in another region, because there will almost always be transitive dependencies on us-east-1. IAM is deployed in us-east-1 and there will always be a transitive dependency on us-east-1

I have never had a production issue in other regions due to a us-east-1 outage. The worst that ever happened was I had to wait to update a Cloudfront distribution because the control plane (based in us-east-1) was down, but the existing configuration continued working fine throughout. I don't know what the architecture of IAM looks like, but somehow it's never suffered a global outage. AWS is really, really good at r…

>I don't know what the architecture of IAM looks like, but somehow it's never suffered a global outage.

Authentication possibly, but the control plane has gone down preventing changes.

Re: AWS us-east-1 down

#217

Post by hx833001 »

throaway87c10f0 wrote: post #164

Mysterious lack of "AWS is bad for the internet because it is so centralized" dialog up in here. edit: for those that would downvote: HN _just_ yesterday: https://news.ycombinator.com/item?id=36295352 https://news.ycombinator.com/item?id=36295305

Too techie and doing things the right way, so CF shouldn’t be successful? Therefore… jealousy? That’s my guess as to why all the hacker news hate.

Re: AWS us-east-1 down

#218

Post by vultour »

kapilvt wrote: post #182
vultour wrote: post #163

Earlier quoted context omitted.

AZs also don't help with natural disasters at all. I believe AWS is the only one doing geographically distributed AZs, for the others it just means different connections and placed somewhere else in the building. edit: turns out AWS is the one with geo distribution, not Azure

Aws azs are also distributed geographically within a region w separate power and network lines. From the docs “ Availability Zones are distinct locations within an AWS Region that are engineered to be isolated from failures in other Availability Zones.”

Ah, you are probably right. I was thinking of the incident a few weeks back where the fire suppression took out multiple AZs, but that was actually GCP.

Re: AWS us-east-1 down

#219

Post by shepherdjerred »

adubashi wrote: post #118

It doesn't matter if your infra is in another region, because there will almost always be transitive dependencies on us-east-1. IAM is deployed in us-east-1 and there will always be a transitive dependency on us-east-1

I thought there was some recent shift on making IAM multi-region?

Re: AWS us-east-1 down

#220

Post by benced »

kroltan wrote: post #60
intsunny wrote: post #28

For those wondering: Currently PDT is 7 hours behind UTC. AWS can do so many things, reporting critical outage updates in UTC is not one of those things.

Semi-related: if you ever feel the need to report times to a global audience, not only make sure to always report the timezone (even if it is the same as the user's), but also use UTC offsets rather than timezone names. Life is too short to remember what each timezone name means and converting to it, UTC offsets are much easier on the mental calculator.

Also report it using IATA time zones (America/Los_Angeles) at least in addition (I'd argue instead of) those abbreviations which are completely unstandardized and not unique.
Post reply on HN