Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AWS's us-east-1 region is experiencing issues

health.aws.amazon.com

Post reply on HN
Print view View original

131–140 of 168 posts

Re: AWS's us-east-1 region is experiencing issues

#131

Post by millzlane »

AitchEmArsey wrote: post #97
bogomipz wrote: post #94

Earlier quoted context omitted.

>"Strict adherence to the "hiring bar" means we fail to bring in good people who aren't desperate enough to act out the cultish LP dance during their interview." What is the "cultish LP dance" here that is weeding good people out? >"My team is hiring for 2-3 people and we are being buried alive without that growth happening sooner - but I can't in good conscience recommend this place to anyone I respect or like." I a…

> What is the "cultish LP dance" here that is weeding good people out? The "culture fit" interview process focuses on leadership principles, so lots of questions like " tell me about a time when you went above and beyond for a customer". Being yourself will get you nowhere, you need to research the questions and the script that is expected of you. > What service does your team work on? I'm a partner-focused SA, so no…

As some dude from Baltimore who has just picked up a second gig there. It seemed to me that these are normal questions asked of you in most interviews. In my daytime position at the first gig I have interviewed technicians and have asked similar questions of them. It's not about culture fit. It's about finding their answer to basic customer service questions. It would make sense that a customer obsessed company would be focused on a set principals they have found success in.

Speaking of the research, the recruiters email you the principals and specifically mention to you to review them and to consider them during the interview process. They even send you a document about the STAR method of interviewing to help you have a smoother interview. To me as a guy from Baltimore who doesn't know anything half the people here do. I don't think the interview process could have been smoother.

Re: AWS's us-east-1 region is experiencing issues

#132

Post by 9wzYQbTYsAIc »

steveBK123 wrote: post #128
halestock wrote: post #17

I can't help but wonder, with the increases in attrition across the industry, are we hitting some kind of tipping point where the institutional knowledge in these massive tech corporations is disappearing? Mistakes happen all the time but when all the people who intimately know how these systems work leave for other opportunities, disasters are bound to happen more and more.

Yes, absolutely. Within my own org of ~50 people, 15% have resigned/contracts ending during Q1 (after 15% in Q4). Of the remaining 85%.. 20% have been around since before COVID / 65% joined during COVID. Of our senior engineers & team leads, 70% have joined in last 6-9 months. Only 3 full time senior engineers with 2 years or more tenure. We've grown during COVID but we've also just burned through people. Turnover ha…

Similar story in smaller orgs, from what I’ve heard - people don’t even bother with the going-away stuff any more.

Re: AWS's us-east-1 region is experiencing issues

#134

Post by 9wzYQbTYsAIc »

sokoloff wrote: post #124
9wzYQbTYsAIc wrote: post #108

Earlier quoted context omitted.

> Amazon is notorious for their frugality and they recently doubled their maximum salary cap to 350k Does that salary cap apply to regular enterprise developers working for HR, Accounting, etc., too? If so, kudos to Amazon.

Why wouldn’t the cap apply to those devs as well?

I can’t imagine why it wouldn’t.

It’s just a little amazing to imagine that people doing the same work in different places of the country have such huge gaps in salary caps. I think the national average for a high-level software engineer is less than $150k per year.

Re: AWS's us-east-1 region is experiencing issues

#135

Post by consumer451 »

consumer451 wrote: post #57
etaioinshrdlu wrote: post #20

Does AWS have a plan to improve this region? Do they acknowledge the problem? It's been a joke for years how bad us-east-1 is.

Nuke the entire site from orbit It's the only way to be sure

Just in case PT's Stasi as a Service company ($PLTR) has a hard time parsing this, I want to make clear that this is a joke based on a quote from Aliens the movie. I am not endorsing anything violent.

It is a joke.

I would delete my parent comment if I could.

Re: AWS's us-east-1 region is experiencing issues

#136

Post by metadat »

9wzYQbTYsAIc wrote: post #132
steveBK123 wrote: post #128

Earlier quoted context omitted.

Yes, absolutely. Within my own org of ~50 people, 15% have resigned/contracts ending during Q1 (after 15% in Q4). Of the remaining 85%.. 20% have been around since before COVID / 65% joined during COVID. Of our senior engineers & team leads, 70% have joined in last 6-9 months. Only 3 full time senior engineers with 2 years or more tenure. We've grown during COVID but we've also just burned through people. Turnover ha…

Similar story in smaller orgs, from what I’ve heard - people don’t even bother with the going-away stuff any more.

It seems like that is more of an artifact of the pandemic and everyone working remote. Are you going to hold a farewell zoom party? I don't think so.

Re: AWS's us-east-1 region is experiencing issues

#137

Post by oblio »

bogomipz wrote: post #121
AitchEmArsey wrote: post #97

Earlier quoted context omitted.

> What is the "cultish LP dance" here that is weeding good people out? The "culture fit" interview process focuses on leadership principles, so lots of questions like " tell me about a time when you went above and beyond for a customer". Being yourself will get you nowhere, you need to research the questions and the script that is expected of you. > What service does your team work on? I'm a partner-focused SA, so no…

Wow, that just seems completely performative. Is "SA" systems admin here?

Solutions Architect, jack of all trades that is a sort of customer consultant and creator of solutions architecture (duh) for customers.

Re: AWS's us-east-1 region is experiencing issues

#138

Post by Lascaille »

9wzYQbTYsAIc wrote: post #70
halestock wrote: post #17

I can't help but wonder, with the increases in attrition across the industry, are we hitting some kind of tipping point where the institutional knowledge in these massive tech corporations is disappearing? Mistakes happen all the time but when all the people who intimately know how these systems work leave for other opportunities, disasters are bound to happen more and more.

Outside of the mega-fang industry, I’m wondering the same thing. The Great Resignation had to have taken a huge toll on regular enterprises. There are probably going to be some unlucky (or lucky, depending on how hardcore they are) people in the position of maintaining aging legacy systems and retrofitting them into the future. COBOL, for example, is becoming a lucrative language for people in the financial and insur…

>The Great Resignation had to have taken a huge toll on regular enterprises.

The Great Resignation was people leaving jobs they didn't want (front of house/service industry/gig) for jobs they did want (career-track jobs) after those jobs resumed hiring again after the pandemic settled down.

Labor force participation went up not down due to 'the great resignation.'

Re: AWS's us-east-1 region is experiencing issues

#139

Post by late2part »

300bps wrote: post #30
operator1 wrote: post #11

What’s up with all of the multi-platform outages lately? Seems abnormal looking at historical data. Are there issues affecting the internet backbone or something? Or just a coincidence?

Important to keep in mind that AWS has 250 services in 84 Availability Zones in 26 regions. This outage is reportedly impacting 5 services in 1 region. For those impacted, pretty terrible. But as a heavy user of AWS, I’ve seen these notices posted multiple times on HN and haven’t been impacted by one yet.

Counterpoint:

us-east-1 is their largest reason, someone told me it's 50% of their revenue

multi-region failover is awfully, awfully expensive.

In my last 7 years I imagine we had ~1 outage a month on average from AWS failures, but who knows if my imagination is accurate.

Re: AWS's us-east-1 region is experiencing issues

#140

Post by saghm »

jrockway wrote: post #91
viraptor wrote: post #27

Earlier quoted context omitted.

Different people have different responsibilities. At Amazon scale, the comms and people doing a deep dive to fix stuff will not be the same.

I'd be totally fine just having alerts and metrics driving the status page. Why involve a human at all? They just get emotional. (I have a data-driven status page for my personal website. If Oh Dear decides my website is down, the status page gets automatically updated. Obviously nobody is ever going to visit status.jrock.us if they are trying to read an article on my blog and it doesn't load, but hey at least I can…

> I'd be totally fine just having alerts and metrics driving the status page. Why involve a human at all?

What happens if the monitoring service goes down or has a bug that causes it to incorrectly report the status as okay? Obviously this won't usually happen at the same time as the actual service goes down, but if it did, would people really believe that?

Post reply on HN