Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Amazon packages pile up after AWS outage spawns delivery havoc

detroitnews.com

Post reply on HN
Print view View original

31–40 of 219 posts

Re: Amazon packages pile up after AWS outage spawns delivery havoc

#31

Post by RyJones »

riffic wrote: post #14
LAC-Tech wrote: post #7

I thought the whole point of AWS that it was a distributed system with no one point of failure? In that case, how did it have an outage?

do they not teach people what a failure domain is anymore? https://en.wikipedia.org/wiki/Failure_domain Clouds aren't magic. They require a certain amount of operational confidence in order to understand that, yes, an entire region can fall out from under you at any time and it's your responsibility to detect and deploy into an unaffected region if possible. edit: Generally, one entire region will not fail. However,…

In my experience, once an item is done done done, it’s out of your hands.

Re: Amazon packages pile up after AWS outage spawns delivery havoc

#32

Post by jaxx »

adventured wrote: post #30
caffed wrote: post #28

I thought this story was about this house x-) "Neighbors in Tennessee city worry as Amazon packages pile up outside home" https://www.youtube.com/watch?v=qVQjEB2sxBw

When a residential house becomes a review farm.

when the house is becoming an amazon warehouse. This is crazy, i can't believe what the home owners thought of so many many boxes coming there.

Re: Amazon packages pile up after AWS outage spawns delivery havoc

#34

Post by bombcar »

cbtacy wrote: post #21
riffic wrote: post #11

Earlier quoted context omitted.

why wouldn't they?

Because it's terrible and has always been terrible? After 2017 I'll never use US-east-1 again. Hell... I should have learned that particular lesson in 2011 but it took two catastrophic failures for me to figure it out. There are numerous threads here on HN covering the topic "why does US-east-1 suck so hard." https://news.ycombinator.com/item?id=13756082 is just one example.

Maybe it’s terrible because they host their own stuff there.

Re: Amazon packages pile up after AWS outage spawns delivery havoc

#36

Post by sithadmin »

riffic wrote: post #14
LAC-Tech wrote: post #7

I thought the whole point of AWS that it was a distributed system with no one point of failure? In that case, how did it have an outage?

do they not teach people what a failure domain is anymore? https://en.wikipedia.org/wiki/Failure_domain Clouds aren't magic. They require a certain amount of operational confidence in order to understand that, yes, an entire region can fall out from under you at any time and it's your responsibility to detect and deploy into an unaffected region if possible. edit: Generally, one entire region will not fail. However,…

Honestly, no. Not frequently enough.

There are also way too many successful, public cloud-native businesses running without any semblance of a Business Continuity or DR Plan among any of their teams. I wish I could name and shame some of the more egregious cases I have seen.

Re: Amazon packages pile up after AWS outage spawns delivery havoc

#37

Post by aschla »

Here in Chicago every item on Amazon is showing an earliest delivery date of Saturday the 11th. Checked an item today I had bought yesterday and delivered overnight, the item is now showing available delivery as Saturday. What a terrible time of the year for this to happen to them.

Re: Amazon packages pile up after AWS outage spawns delivery havoc

#38

Post by deathanatos »

riffic wrote: post #14
LAC-Tech wrote: post #7

I thought the whole point of AWS that it was a distributed system with no one point of failure? In that case, how did it have an outage?

do they not teach people what a failure domain is anymore? https://en.wikipedia.org/wiki/Failure_domain Clouds aren't magic. They require a certain amount of operational confidence in order to understand that, yes, an entire region can fall out from under you at any time and it's your responsibility to detect and deploy into an unaffected region if possible. edit: Generally, one entire region will not fail. However,…

> do they not teach people what a failure domain is anymore?

Empirically? No.

But seriously, instead of making every dev team consuming an AWS product hire the extra engineers required to build a system that spans multiple failure domains — which lets be honest, companies won't — why hasn't Amazon just hired the engineers required to do it for me?

> Generally, one entire region will not fail.

Laughs in global failures.

Re: Amazon packages pile up after AWS outage spawns delivery havoc

#39

Post by haggy102 »

imwillofficial wrote: post #27
riffic wrote: post #11

Earlier quoted context omitted.

why wouldn't they?

If their stuff was all super resilient and multi AZ, it would be easier to place blame on the customer

Multi AZ != Multi region. AWS has many services that run in multiple regions but some of their services are not.

Re: Amazon packages pile up after AWS outage spawns delivery havoc

#40

Post by treis »

sneak wrote: post #16
andrewguenther wrote: post #15

Earlier quoted context omitted.

Region outages happen, which is why the guidance is to build in multiple regions, but even Amazon sometimes doesn't take their own advice. Sometimes the reason is good, sometimes it isn't.

I thought the guidance was to be multi-AZ, as an AZ is the failure boundary?

Software bugs recognize no boundary
Post reply on HN