What’s up with all of the multi-platform outages lately? Seems abnormal looking at historical data. Are there issues affecting the internet backbone or something? Or just a coincidence?
Increase in attrition across the industry. A lot of institutional knowledge in these massive tech corporations is disappearing and we're starting to reach the tipping point.
AWS's us-east-1 region is experiencing issues
71–80 of 168 posts
Re: AWS's us-east-1 region is experiencing issues
#72I can't help but wonder, with the increases in attrition across the industry, are we hitting some kind of tipping point where the institutional knowledge in these massive tech corporations is disappearing? Mistakes happen all the time but when all the people who intimately know how these systems work leave for other opportunities, disasters are bound to happen more and more.
Disc: Googler opinions are my own. I've been an SRE on a tier 1 GCP product for over three years this is not the case. In my experience, our systems have only gotten more reliable and easier to understand since I joined. It's not like there are only a few key knowledge holders that single handedly prevent outages from happening. In reality, you don't need to know shit about how a system works internally to prevent ou…
Google doesn’t have nearly as hard a time retaining good people as Amazon does.
Re: AWS's us-east-1 region is experiencing issues
#73Earlier quoted context omitted.
Just like the tech priests in Warhammer 40k, keeping occult old engineering, thatno one could build anymore, running
If we want to normalize letting long term support people call themselves tech priests I'd very much appreciate it. "What were your duties at your last position?" "Performing the daily ministrations and singing the praise of the machine god."
Re: AWS's us-east-1 region is experiencing issues
#74Earlier quoted context omitted.
Not familiar with 40k. Was it a similar idea to nuclear-power-as-religion from Foundation?
Or how about Anathem, with the Ita class doing computer things and nuclear materials cared for by a select group?
Scientists only get to talk to the public every 100 years or something, wasn’t it?
IT was never allowed to talk to scientists.
Seemed like a modernist idea, even at the time of publishing.
Re: AWS's us-east-1 region is experiencing issues
#75I can't help but wonder, with the increases in attrition across the industry, are we hitting some kind of tipping point where the institutional knowledge in these massive tech corporations is disappearing? Mistakes happen all the time but when all the people who intimately know how these systems work leave for other opportunities, disasters are bound to happen more and more.
You're right, but that's been true since the beginning of the tech boom (but isn't exclusive to tech) when no one works for a place for several decades. Companies weather this in different ways but attrition has always been around. What's causing people to believe that the latest round of attrition is any different?
The Great Recession
The Great Resignation
The Great Dying (due to COVID-19)
Re: AWS's us-east-1 region is experiencing issues
#76Earlier quoted context omitted.
Not familiar with 40k. Was it a similar idea to nuclear-power-as-religion from Foundation?
Not far off. The “golden age” of humanity was shattered long ago, with the mortal wounding of the god emperor, and knowledge of most of the greatest technology was lost.Millennia later, a cult has grown up that both worships and maintains technology as having machine spirits, which are somehow linked to the machine god itself. That god may or may not be the same or related to the god emperor of mankind, depending on…
It seems like both sides are fine to have them be reconciled, but it's an important narrative gadget that can be used to get humanity to fight itself in-universe.
Also interesting is that humanity's "lost" technological progress seems to eclipse some of the other races in the W40K narrative, with even the Tau (space dwarves with robots) and Eldar (space elves with crystals) freaking out when humanity brings giant robots, because the sheer physical impracticality of a gigantic human shaped robot is noted, with nobody aware of how they continue to work.
Re: AWS's us-east-1 region is experiencing issues
#77Maybe the reason AWS keeps going down is because they run all their stuff on-prem...
Re: AWS's us-east-1 region is experiencing issues
#78Earlier quoted context omitted.
Disc: Googler opinions are my own. I've been an SRE on a tier 1 GCP product for over three years this is not the case. In my experience, our systems have only gotten more reliable and easier to understand since I joined. It's not like there are only a few key knowledge holders that single handedly prevent outages from happening. In reality, you don't need to know shit about how a system works internally to prevent ou…
I’ll be honest. My external impression of Amazon and Google could not be more distant in this regard. Google doesn’t have nearly as hard a time retaining good people as Amazon does.
Strict adherence to the "hiring bar" means we fail to bring in good people who aren't desperate enough to act out the cultish LP dance during their interview. Hiring new grads seems to be the only area where growth is not stalling - but that can obviously only help so much.
My team is hiring for 2-3 people and we are being buried alive without that growth happening sooner - but I can't in good conscience recommend this place to anyone I respect or like.
Re: AWS's us-east-1 region is experiencing issues
#79Earlier quoted context omitted.
Just like the tech priests in Warhammer 40k, keeping occult old engineering, thatno one could build anymore, running
If we want to normalize letting long term support people call themselves tech priests I'd very much appreciate it. "What were your duties at your last position?" "Performing the daily ministrations and singing the praise of the machine god."
Re: AWS's us-east-1 region is experiencing issues
#80What’s up with all of the multi-platform outages lately? Seems abnormal looking at historical data. Are there issues affecting the internet backbone or something? Or just a coincidence?