Live data from Hacker News

AWS's us-east-1 region is experiencing issues

health.aws.amazon.com

81–90 of 168 posts

Re: AWS's us-east-1 region is experiencing issues

#81
post #51

Earlier quoted context omitted.

You're right, but that's been true since the beginning of the tech boom (but isn't exclusive to tech) when no one works for a place for several decades. Companies weather this in different ways but attrition has always been around. What's causing people to believe that the latest round of attrition is any different?

I'd speculate that perhaps more senior people are moving, and/or a greater overall rate of attrition combined with much more complex technologies and organisations. In other words, it might be harder to become good at jobs now, and fewer people stick with them. Just a hunch but definitely seems to be where the incentives point with loyalty penalties and tech bloat.

In my experience, education certainly seems to not have kept up with computing, at least in terms of having massive academic-industry partnerships like a doctors residency or a trade apprenticeship .

I’d definitely agree that it is probably harder to become good in older organizations - the technologies are probably generations behind the current state of the art and the learning curves are high for those older technologies.

Just thinking through keyboard, but it’s probably reaching the point where enterprises need to evaluate aqui-hire or outsourcing entire development departments due to attrition, due to the incentives to leave for regular employees.

Re: AWS's us-east-1 region is experiencing issues

#83
post #8

This is why you are strongly urged not to rely on one region or AZ.

I would strongly urge not using us-east-1 -- of all the regions we're in, it's by far the most problematic. Use us-east-2 if you need good latency to the East Coast.

Not sure if it's still the case, but when I was there us-east-1 was a SPOF for some services world wide. I think if dynamodb went down in the region it was a big, big issue.

Re: AWS's us-east-1 region is experiencing issues

#84
post #76
post #55

Earlier quoted context omitted.

Not far off. The “golden age” of humanity was shattered long ago, with the mortal wounding of the god emperor, and knowledge of most of the greatest technology was lost.Millennia later, a cult has grown up that both worships and maintains technology as having machine spirits, which are somehow linked to the machine god itself. That god may or may not be the same or related to the god emperor of mankind, depending on…

FWIW hardline tech priests view the machine god as separate from the emperor. Hardline imperium officials view the emperor as separate from the machine god. The official party line is that the emperor and the machine god are the same, with the emperor perhaps being an avatar. It seems like both sides are fine to have them be reconciled, but it's an important narrative gadget that can be used to get humanity to fight…

> gigantic human shaped robot is noted, with nobody aware of how they continue to work

BattleTech had something similar: it was considered cheaper to keep replacing humans than to replace the mechs, because hardly anyone knew how to repair or build mechs.

Re: AWS's us-east-1 region is experiencing issues

#85

I can't help but wonder, with the increases in attrition across the industry, are we hitting some kind of tipping point where the institutional knowledge in these massive tech corporations is disappearing? Mistakes happen all the time but when all the people who intimately know how these systems work leave for other opportunities, disasters are bound to happen more and more.

No

Re: AWS's us-east-1 region is experiencing issues

#86

Earlier quoted context omitted.

You're right, but that's been true since the beginning of the tech boom (but isn't exclusive to tech) when no one works for a place for several decades. Companies weather this in different ways but attrition has always been around. What's causing people to believe that the latest round of attrition is any different?

> What's causing people to believe that the latest round of attrition is any different? The Great Recession The Great Resignation The Great Dying (due to COVID-19)

> The Great Dying (due to COVID-19)

Repeating this wise comment: https://news.ycombinator.com/item?id=23769427

The COVID death counts are hopelessly over-counted. This is why there's a cottage industry of people pointing out things like "COVID deaths" which mysteriously also suffered from being murdered, or drug overdoses, or undiagnosed leukaemia.

Then you get into the problem of care homes being authorised to report COVID deaths without any testing or formally trained opinion at all.

Re: AWS's us-east-1 region is experiencing issues

#87
post #85

I can't help but wonder, with the increases in attrition across the industry, are we hitting some kind of tipping point where the institutional knowledge in these massive tech corporations is disappearing? Mistakes happen all the time but when all the people who intimately know how these systems work leave for other opportunities, disasters are bound to happen more and more.

No

That’s good to hear. How wide is your scope of experience?

Re: AWS's us-east-1 region is experiencing issues

#88
post #72

Earlier quoted context omitted.

I’ll be honest. My external impression of Amazon and Google could not be more distant in this regard. Google doesn’t have nearly as hard a time retaining good people as Amazon does.

From within AWS, it just feels like we push people too hard. Service teams are too small relative to their goals, sales teams have unrealistic growth targets and also double as support for needy/incompetent customers, and professional services have billable hour requirements which are as high as any major consultancy and with additional pre-sales support expectations. Strict adherence to the "hiring bar" means we fai…

I don't work anywhere as big as Amazon/AWS but it is still a "big tech engineering firm" and we're experiencing all the same problems.

Re: AWS's us-east-1 region is experiencing issues

#89
post #72

Earlier quoted context omitted.

I’ll be honest. My external impression of Amazon and Google could not be more distant in this regard. Google doesn’t have nearly as hard a time retaining good people as Amazon does.

From within AWS, it just feels like we push people too hard. Service teams are too small relative to their goals, sales teams have unrealistic growth targets and also double as support for needy/incompetent customers, and professional services have billable hour requirements which are as high as any major consultancy and with additional pre-sales support expectations. Strict adherence to the "hiring bar" means we fai…

> From within AWS, it just feels like we push people too hard.

Sounds like regular skeleton-crew enterprise IT.

Re: AWS's us-east-1 region is experiencing issues

#90
post #80

What’s up with all of the multi-platform outages lately? Seems abnormal looking at historical data. Are there issues affecting the internet backbone or something? Or just a coincidence?

A handful of large-traffic sites have recently, and relatively suddenly, started blocking traffic from a large region. That's a major change in flow.

Could you be more specific?
Post reply on HN