Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Technical Interviews Reject the Wrong Engineers

fagnerbrack.com

Post reply on HN
Print view View original

61–70 of 111 posts

Re: Technical Interviews Reject the Wrong Engineers

#61

Post by sanderjd »

tptacek wrote: post #34
HarHarVeryFunny wrote: post #26

The main goal of hiring someone should be to assess how well they can do the job you are hiring for, and for anyone with job experience (i.e. not fresh out of college) the best indicator of that is what have they previously achieved (especially in more recent years). How well someone can solve a whiteboard challenge or brainteaser is irrelevant unless you are hiring someone to solve 10min whiteboard challenges. Of co…

Experience-based interviews are a fantastic way to select for candidates who have "failed up" through a long series of jobs. The underlying dynamic is that it usually takes more than a year to sever a technical employee; you can faceplant in a role and still wind up with a resume improvement.

I hear this a lot, but man, I just really don't think so. First of all, 1-year stints come off very poorly in this kind of discussion. (I would say that a bias against people with a bunch of short stints would be a failure mode of "experience-based interviews" rather than the failure mode you're describing.) But also, I have had these discussions with (in my judgement) both "failed up" candidates and "tons of valuable experience" candidates, and I just don't have this experience that it's difficult to differentiate.

I'm fully aware that according to the internet, there is an epidemic of bullshit artists who can go deep on the architecture and tradeoffs and their contribution to the things they worked on, without having actually contributed to those things, but I dunno, the narrative just doesn't jibe with my anecdotal experience.

My only uncertainty here is that I do think I have been very fortunate in the people I have worked with in my career, so I might just be getting lucky. But I have truly never worked with someone who is hired largely on the basis of a strong resume, but genuinely can't grok fizzbuzz, or whatever the contemporary equivalent of that is.

I also recognize that you ran a real company doing real work on this, so I'm generally inclined to defer to your wisdom...

You can tell that I'm very torn on this, because the conventional wisdom is so strongly against my perspective on it, and I generally put a good deal of weight on conventional wisdom. But man, I dunno, it all really feels like an inertia thing that I increasingly question the original foundations of. Sometimes the emperor actually doesn't have any clothes on...

Re: Technical Interviews Reject the Wrong Engineers

#62

Post by robofanatic »

andrewstuart wrote: post #22

Even harder with AI. How do you assess developers when AI makes it possible to create software without even knowing the language?

Give an abstract requirement and access to your AI tool. Ask the candidate to create a working solution and review the AI generated code. requirement analysis and code review are now the primary skills for developers.

Re: Technical Interviews Reject the Wrong Engineers

#63

Post by voakbasda »

mathisfun123 wrote: post #21
zipy124 wrote: post #7

The main problem is good engineers have no need to sit through your 12 step process. It actively selects only for the most desperate or money driven people (if you pay very well).

> The main problem is good [doctors] have no need to sit through your 12 [years of school]. It actively selects only for the most desperate or money driven people (if you pay very well). do you agree with this?

Absolutely. Witnessed it directly in the form of med students paying other people to take their tests for them.

Re: Technical Interviews Reject the Wrong Engineers

#64

Post by cmrdporcupine »

Aurornis wrote: post #59
sanderjd wrote: post #44

Earlier quoted context omitted.

Yeah this is my "natural" interviewing style. Like, I have a resume, I'm talking to a person, my natural curiosity about the person and their work leads me to exactly the kind of conversation you're describing. Then my hire / no-hire intuition is basically "am I impressed with them after that conversation?". But then I've also read a huge amount about interviewing "correctly" over the years, probably starting with th…

Hiring based on gut feeling about how impressive the candidate feels can be misleading when you only do a little hiring. It can work for small samples sizes if you have a strong front end filter or you are primarily getting candidates through trusted referrals. Then one day you encounter a candidate who is great at impressing people. They leave you feeling excited with the possibility of working with them. You feel d…

Or there's potentially this: that the skills we think are universal and transferable ... are not.

That our industry is so bespoke and different and non-standard around tech stack organization and work culture between shops that in fact someone could have been performing top tier at their previous jobs and just completely not function in your workplace.

And because we have a recency and confirmation bias around your own positions, we treat their failure to thrive in our environment as evidence of incompetence.

When it's just really a bad fit -- one you can't really shake out in a verbal conversation or a whiteboard coding exercise.

Or there could even be something wrong with the way our own workplace is organized. One that we're used to working around, but the new hire is not.

There's so much broken everywhere I've worked. When I see a misfit I generally assume it's a combination of factors, not a person lying or being incompetent (though yes, I've seen that, too)

Re: Technical Interviews Reject the Wrong Engineers

#65

Post by newtonianrules »

dangus wrote: post #50
newtonianrules wrote: post #14

Earlier quoted context omitted.

I don’t want to put my future coworker through six rounds of interviews. If it takes more than three rounds + a phone screen to figure out if someone is a good fit then the process is broken.

Depends how long the rounds are. 6 rounds of 20 minutes is only 2 hours. If you think that’s unreasonable, please go ahead and add a few fire sauce packets to the bag for me.

How many interviews have you been on that a round is 20 minutes?

Re: Technical Interviews Reject the Wrong Engineers

#66

Post by kadhirvelm »

root-parent wrote: post #13
kadhirvelm wrote: post #11

We’ve added collaboration and communication as big facets in our hiring loop rubrics, and reduced the complexity of our questions. We also tell candidates this ahead of time - been honestly working pretty well so far, it indexes more on those soft skills and gives us just enough insight into the hard skills to make a call. We’ve also been increasingly finding very few candidates know how to solve a complex question t…

>> few candidates know how to solve a complex question these days without LLM support lol Please tell me you are joking.

I wish, but maybe it’s more fair to say how people think has changed. It’s like for these complex questions, they can break the problem down quickly into smaller functions, but implementing the functions is slow? Sometimes a non-starter? Idk, it’s different. Our old ways of interviewing aren’t working the same

Re: Technical Interviews Reject the Wrong Engineers

#67

Post by andrewstuart »

robofanatic wrote: post #62
andrewstuart wrote: post #22

Even harder with AI. How do you assess developers when AI makes it possible to create software without even knowing the language?

Give an abstract requirement and access to your AI tool. Ask the candidate to create a working solution and review the AI generated code. requirement analysis and code review are now the primary skills for developers.

Interesting.

I don’t think code review of AI is important at all - specially given the developer might not even know the language, in which case it’s irrelevant.

I think ability to build checking, verification, linting, testing and cross LLM code review mechanisms is important, which ensure that fast changing or unpredictably changing ai code has consistent behavior and is security checked.

Re: Technical Interviews Reject the Wrong Engineers

#68

Post by tptacek »

sanderjd wrote: post #61
tptacek wrote: post #34

Earlier quoted context omitted.

Experience-based interviews are a fantastic way to select for candidates who have "failed up" through a long series of jobs. The underlying dynamic is that it usually takes more than a year to sever a technical employee; you can faceplant in a role and still wind up with a resume improvement.

I hear this a lot, but man, I just really don't think so. First of all, 1-year stints come off very poorly in this kind of discussion. (I would say that a bias against people with a bunch of short stints would be a failure mode of "experience-based interviews" rather than the failure mode you're describing.) But also, I have had these discussions with (in my judgement) both "failed up" candidates and "tons of valuabl…

First, they're not "1 year stints". It takes about a year to manage someone out of a role once you've decided you need to do that. Nobody makes that decision on day 1 of a report's tenure.

Second, how well the "stint" comes across in the interview is a question of how skilled the candidate is at talking. You can make anything sound like anything in an interview. The interviewer has no reliable way of checking.

Re: Technical Interviews Reject the Wrong Engineers

#69

Post by sanderjd »

Aurornis wrote: post #59
sanderjd wrote: post #44

Earlier quoted context omitted.

Yeah this is my "natural" interviewing style. Like, I have a resume, I'm talking to a person, my natural curiosity about the person and their work leads me to exactly the kind of conversation you're describing. Then my hire / no-hire intuition is basically "am I impressed with them after that conversation?". But then I've also read a huge amount about interviewing "correctly" over the years, probably starting with th…

Hiring based on gut feeling about how impressive the candidate feels can be misleading when you only do a little hiring. It can work for small samples sizes if you have a strong front end filter or you are primarily getting candidates through trusted referrals. Then one day you encounter a candidate who is great at impressing people. They leave you feeling excited with the possibility of working with them. You feel d…

I'm certain that this is true. It certainly sounds true.

But honestly, does anyone here have experience with doing this kind of interviewing at scale, for experienced software engineering roles? I've been in the industry for coming up on a couple decades, and I have been involved in doing lots of interviews at times, and in that whole period of time, at every company where I've worked, we did the standard multi-round whiteboard / coderpad interviews. Do other folks here actually have recent experience "hiring at scale" in this industry, with a process focused on candidates' experience? Who is doing that?

And also, the question is not "are there ever false positives?" or even "is this biased toward a certain kind of false positive?". Nobody thinks any way of hiring is perfect. Even ignoring bullshit artists, sometimes very competent people just aren't a good fit for the actual demands of a particular job. The question is one of tradeoffs. Are the failure modes and biases of a particular process worse than those of another one?

To me, the current standard process comes at an enormous cost. At any job I've ever had, every time I think that maybe my time there has run its course, I immediately think, ugh, but I'll have to go through the f**ing interview process. I'm not a person who does research on this, so I don't have data or anything, but I must not be alone in this, and I think it is likely a meaningful friction in the job market. (Which, I guess is good for employers, so it probably makes sense that they like the status quo!)

Re: Technical Interviews Reject the Wrong Engineers

#70

Post by SatvikBeri »

Every interview method has some glaring flaws, and I find you mostly have a choice of which flaws you pick.

On my current team, I care a lot about the ability to write fast code. The most important part of my process is a take-home designed to take 2 hours where the main goal is to solve a relatively easy problem as performantly as possible. Answers have varied from 0.2ms – 50ms.

Take-homes have some obvious disadvantages, but overall I find they're better at finding the people we're looking for than just about every other method. But I'm at a small company, hiring for a fairly specialized team. If the situation was different (e.g. I needed to hire 50 people/year) I'd use a much more standard process.

Post reply on HN