Hire-to-fire at Amazon India? • Hacker News Board

Re: Hire-to-fire at Amazon India?

#331

Post by MattGaiser » Sun, Jun 20, 2021, 7:15 PM UTC

nkozyra wrote: post #262

grumple wrote: post #260

Earlier quoted context omitted.

I'm in the US and have been approached by multiple recruiters who explained the the first part of hiring was a leetcode-like online assessment, and a later part of the process was a full day onsite (which I understood to be leetcode-like, or systems design questions, just like other big tech companies). I did not actually go through the process. They might not be using leetcode.com itself, but they are doing similar…

Well then the point about leetcode doesn't mean much here. Yes, most companies do coding assessments, but that doesn't mean it's ironic that the comment was posted on leetcode itself. My experience with Amazon had no self guided coding assessment, it involved a human guiding you through a problem or multiple problems.

Hit and miss for me with Amazon recruiters. Some have a Hacker Rank and some say I can skip it.

Re: Hire-to-fire at Amazon India?

#332

Post by dnautics » Sun, Jun 20, 2021, 7:16 PM UTC

jjtheblunt wrote: post #300

salil999 wrote: post #18

Former AWS engineer here. I worked on a pretty critical product in AWS (big AWS service with lots of traffic) and I can safely say that it's totally up to your manager and pre-existing conditions which make up the job. My manager was great as a person but would always lack in my career-oriented goals (bigger projects, promotions, etc) But what really sucked for me was the pre-existing conditions. Our on-call was pret…

Current Amazon engineer: it's far and away the most incompetently run bureaucracy with self-defeating dysfunctions forced on the huge number of layers. The recent "leaks" to Business Insider, pointed out to me by coworkers, are exactly what we see. Everything the parent comment mentioned is exactly what we see. Laughably, our middle manager berates us as incapable noobs, entirely unaware that some of us at least actu…

well the good news (for amazon) is that I used to work at a place where I was trying to build a cloud. And management suffered from EXACTLY THE SAME PROBLEMS (though for different structural reasons, there was no "build a feature" incentive). And then we hired someone who used to work on amazon cloud who came with rave reviews, even though he completely failed my interview, which basically tested "do you read the spec?". Then I told him to take notes while I was onboarding him, which he didn't do, and then I quit, because of the management problems.

> the engineering quality is so shoddy

What stuns me is just how well AWS works. I mean, did you ever try to use Azure in 2015? Hell, even GCP was worse in that era. Now, it's on-par, but IMO "more confusing" which beggars belief considering how confusing AWS is.

Re: Hire-to-fire at Amazon India?

#333

Post by awsthro00945 » Sun, Jun 20, 2021, 7:17 PM UTC

vlovich123 wrote: post #315

awsthro00945 wrote: post #180

Earlier quoted context omitted.

>Our on-call was pretty bad (40-60 tickets a week) and there was very little investment being put in to improve it. We had a lot of little scripts here and there which would solve extremely specific situations but no focus was ever put on in building a general framework or trying to reduce the ticket count. AWS engineer here and I confirm everything you say, but this quote really struck home with me. The thing I've n…

Can you share your knowledge or reading materials for how to reduce on-call load? I’ve worked at a number of big companies but all the problems driving the oncall load seemed, at best, domain specific if not application specific with highly variable fix times and unpredictable occurrence (eg started becoming more of a problem due to unrelated change X). As a result each team has to decide the cost of fixing the pain…

I don't have any particular reading material, and my example of the on-call load at AWS that I'm referring to is probably very basic to most people.

On my team at AWS, leadership has given specific instruction that we do not believe in on-call runbooks or automation to triage issues, for example. Leadership's reasoning for this is that they think runbooks prevent engineers from applying personal judgement, and every single issue should be handled manually by an engineer on an ad-hoc basis.

This leads to a significant amount of on-call time and cognitive load spent doing stuff like verifying the most basic of issues. Even if you have seen the same issue come up for the 1000th time, and even though the previous 999 times it came up the answer was always the same, leadership still insists that the on-call engineer go through a full ad-hoc process of investigating the issue "just to be sure" that this time isn't different.

It's a similar situation with documenting our integration guidance for other teams. Our leadership insists that any documented guidance be vague, and that whenever another team wishes to integrate with our software they must schedule meetings with us to discuss even the most basic of design questions. I'm talking very simple stuff like "should you use the HTTPS endpoint to communicate with our service?" where the answer is "yes" 99.99% of the time, and could easily be included in some documentation. But leadership insists that we spend multiple hours per week in meetings to discuss this just in case that 0.01% design comes up.

Re: Hire-to-fire at Amazon India?

#334

Post by kondu » Sun, Jun 20, 2021, 7:17 PM UTC

chewzerita wrote: post #297

steelframe wrote: post #143

Earlier quoted context omitted.

I'm at the point of my career where it's all about the manager, the hours, the comp, and the kind of work I'll be doing. I no longer care as much about the title or level. In fact if the level is lower, great -- I'll have an even easier time getting high perf scores. These days, when I negotiate for a new job in Big Tech, I first and foremost maximize base pay. That's [1] immediate (no waiting for vest or anything),…

> After that, I make sure I won't be carrying a pager that can ever "go off" in the middle of the night. That's a non-negotiable for me at this point in my career. I'm currently in undergrad so starting my actual "career" still seems far enough off for me, but serious question how do people actually accept that? Will I have to accept that when I apply for junior/entry level positions? I don't think I'm asking too muc…

Unfortunately, you don't always know if on-call will be involved until you start at the job. Often many companies don't even hire you with a particular role in mind, instead you are matched to a team after you're signed on. (Especially for new grads). And sometimes on-call is introduced in an existing role, it can be difficult to refuse, especially for people in more junior roles.

But definitely ask about on-call when interviewing, in case they say they have it you can bail out before you sign.

Re: Hire-to-fire at Amazon India?

#335

Post by doggodaddo78 » Sun, Jun 20, 2021, 7:19 PM UTC

EliRivers wrote: post #282

awsthro00945 wrote: post #180

Earlier quoted context omitted.

>Our on-call was pretty bad (40-60 tickets a week) and there was very little investment being put in to improve it. We had a lot of little scripts here and there which would solve extremely specific situations but no focus was ever put on in building a general framework or trying to reduce the ticket count. AWS engineer here and I confirm everything you say, but this quote really struck home with me. The thing I've n…

Related to part of what you said. I've worked in a half dozen different industries; I've worked in little companies with a half-dozen employees and globe spanning companies with tens of thousands and employees. They all think that they have special unique problems that nobody else has - 90% of it is the same problem I've seen in other companies in different industries, with different industry specific acronyms and wo…

"NIH" duplication of effort with "IH" bugs.

SMH. Will they ever learn?

Re: Hire-to-fire at Amazon India?

#336

Post by mavelikara » Sun, Jun 20, 2021, 7:23 PM UTC

notacoward wrote: post #214

ed_elliott_asc wrote: post #198

Earlier quoted context omitted.

Everyone should be testing in prod, in that you release code and see metrics and monitoring to show that everything is working. Testing in production is not going “let’s see if this will work” it is “we will release and validate that everything is working as expected” People need to get over the old school cowboys who jump on prod to see if something works.

Yes, everyone should release code and watch metrics etc. but I think that's at the very edge of what "testing" encompasses. Between model checking, traditional forms of testing, and shadow-traffic testing (which can test higher per-server load than prod), finding something after deploy should be like a parachute failure. Yes those happen, yes there should be a reserve, but if it happens more than once in a blue moon…

Tangential, where can I learn more about shadow-traffic testing? Books, blogs, tools etc.

Re: Hire-to-fire at Amazon India?

#337

Post by ipaddr » Sun, Jun 20, 2021, 7:24 PM UTC

kablow wrote: post #209

salil999 wrote: post #18

Former AWS engineer here. I worked on a pretty critical product in AWS (big AWS service with lots of traffic) and I can safely say that it's totally up to your manager and pre-existing conditions which make up the job. My manager was great as a person but would always lack in my career-oriented goals (bigger projects, promotions, etc) But what really sucked for me was the pre-existing conditions. Our on-call was pret…

I'm in non-AWS and things aren't any better here. It is very much dependent on the manager and team, and I've been optimistic (~3 years now), but it gets harder as time goes on. > I'm just not 100% sure about the whole PIP scene. Our service was extremely critical and we were extremely understaffed. So I don't think it applied to anyone in our org but I know of other teams who would have no issues in taking in a fres…

Amazon has a lot of money to waste. It is cheaper to hire someone so they can fire them in a few months to keep the rest of the staff scared enough to overwork themselves so they can understaff.

They play a good game.

Re: Hire-to-fire at Amazon India?

#338

Post by tolbish » Sun, Jun 20, 2021, 7:24 PM UTC

awsthro00945 wrote: post #309

SheinhardtWigCo wrote: post #291

Earlier quoted context omitted.

I don’t get this. In this market, if you’re good, why take a job with any on-call work? It’s thankless, shitty work. If you can deliver features, that’s good enough to get paid extremely well at other Amazon-scale companies with dedicated SRE teams. What’s making you stay?

Because changing jobs is really fucking difficult. I'm probably one of the biggest proponents of "quit your job, you deserve better" that you'll ever find, but even I have to admit that finding a new job is ridiculously hard. Even in "this market", even if you're a top engineer, it's still ridiculously hard to even get an interview, let alone get hired. There is only a limited amount of companies that will pay at the…

It sounds like the real reason is that the money is worth it.

Re: Hire-to-fire at Amazon India?

#339

Post by hintymad » Sun, Jun 20, 2021, 7:25 PM UTC

kaesar14 wrote: post #326

hintymad wrote: post #292

Earlier quoted context omitted.

What I meant was that Netflix built their platforms and services on top of AWS services, just like AWS teams. Yet AWS teams have brutal oncalls, while Netflix teams enjoy great work-life balance.

But that’s because the AWS teams are presumably shouldering the burden, then.

I’m sure you’re right for some services, especially infra ones, such as EC2. Some other services, though, should be built on top of EC2, EBS, Lambda, S3, and etc, in which case Netflix and AWS teams use the same infra, yet Netflix internal services require much less oncall

Re: Hire-to-fire at Amazon India?

#340

Post by YZF » Sun, Jun 20, 2021, 7:25 PM UTC

vlovich123 wrote: post #315

awsthro00945 wrote: post #180

Earlier quoted context omitted.

>Our on-call was pretty bad (40-60 tickets a week) and there was very little investment being put in to improve it. We had a lot of little scripts here and there which would solve extremely specific situations but no focus was ever put on in building a general framework or trying to reduce the ticket count. AWS engineer here and I confirm everything you say, but this quote really struck home with me. The thing I've n…

Can you share your knowledge or reading materials for how to reduce on-call load? I’ve worked at a number of big companies but all the problems driving the oncall load seemed, at best, domain specific if not application specific with highly variable fix times and unpredictable occurrence (eg started becoming more of a problem due to unrelated change X). As a result each team has to decide the cost of fixing the pain…

I'm not the parent but lemme chime in on this topic. It's pretty simple, if you don't build crappy software you won't get a heavy on-call load. You're really asking how to build great software. Build strong teams, with experienced people, follow good practices, reward quality and stability and not features or lines of code, reduce complexity, etc. etc. I've worked on software used by millions of people with a very low problem rate and then I worked on software used by hundreds of people where nothing ever works. Often in the latter the team, through lack of experience or ability, assumes that this is just the way all software is. There's plenty of examples of widely used software systems that are generally quite reliable and well built, and there's plenty of examples of stuff that's garbage, held together by duct tape, works by chance.