Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Last Technical Interview

steve-yegge.medium.com

Post reply on HN
Print view View original

51–60 of 291 posts

Re: The Last Technical Interview

#51

Post by tptacek »

dpark wrote: post #48
tptacek wrote: post #43

Earlier quoted context omitted.

Adding take-home problems to a traditional 4-6 hour interview loop is odious. But the "way more than 4 hours" thing smuggles in a premise: that every candidate should be able to finish the challenge in the allotted time. But candidates with greater aptitude or conversance with the problem domain will complete work sample tests faster than candidates without, and selecting for those candidates is the point of hiring q…

This is theoretically true but it’s also rife with misaligned priorities. The people putting together these take home assignments have little incentive to ensure that they can be completed by a competent engineer in the allotted time. The engineers completing these assignments are definitely incentivized to underreport how long they spent on the assignment. With AI coding this is also largely useless. These “build th…

We don't ask or check how long candidates take. You're a professional, we give you a challenge, you can decide (up front, 30 minutes in, whatever) whether you're likely to be able to finish in the budgeted time. Maybe you can't because you've got a lunch date and don't have the contiguous block, and want to do it in chunks. Fine by us.

Again, the underlying smuggled premise here is that candidates have to finish the work sample. No they don't. In fact, that's a strong sign it's not an effective work sample. A test that everybody passes isn't a real test; it's just a hazing ritual.

Re: The Last Technical Interview

#52

Post by tptacek »

syndacks wrote: post #11
thaumasiotes wrote: post #10

Earlier quoted context omitted.

> I gave the feedback at one Google interview that they should send Google employees through to see how many get hired. Good to see they basically tried that. They did, but not with the intention of doing anything about the problem. This is a question of reliability , the conceptual 'correlation' of a measurement instrument with itself when measuring the same thing. Reliability is one of two major concepts in psychom…

Serious question, tell me what you think of using IQ tests to hire SWEs? Should we just do that instead?

He said standardized test, not standard general cognitive test.

Re: The Last Technical Interview

#53

Post by dpark »

tptacek wrote: post #51
dpark wrote: post #48

Earlier quoted context omitted.

This is theoretically true but it’s also rife with misaligned priorities. The people putting together these take home assignments have little incentive to ensure that they can be completed by a competent engineer in the allotted time. The engineers completing these assignments are definitely incentivized to underreport how long they spent on the assignment. With AI coding this is also largely useless. These “build th…

We don't ask or check how long candidates take. You're a professional, we give you a challenge, you can decide (up front, 30 minutes in, whatever) whether you're likely to be able to finish in the budgeted time. Maybe you can't because you've got a lunch date and don't have the contiguous block, and want to do it in chunks. Fine by us. Again, the underlying smuggled premise here is that candidates have to finish the…

You are assuming the assignment is reasonable and the candidate is lacking if they cannot complete in the expected time. And for all I know that’s entirely true for you. What I know is that I’ve seen assignments from others where the assignment scope was unreasonable for the allotted time. And for those teams, the filter becomes not so much “is this person capable” but “is this person willing to put up with our shit”, and the teams likely don’t even realize that’s what they’ve done (because they also “don't ask or check how long candidates take”).

> Again, the underlying smuggled premise here is that candidates have to finish the work sample. No they don't. In fact, that's a strong sign it's not an effective work sample. A test that everybody passes isn't a real test; it's just a hazing ritual.

Now this is an interesting take. Usually when people talk about these take home assignments, they talk about assessing the quality of the work. How good is the design? How is the coding? Is it efficient/elegant/whatever?

Here you take a much different approach, saying that the completion itself is the filter. If one person completes your assignment in the allotted 2 hours and another needs 12 but never tells you that, do you not care about that discrepancy?

Re: The Last Technical Interview

#55

Post by kevin_nisbet »

tptacek wrote: post #13

The gold standard in hiring qualification is work-sample testing. It works fine. You do not need to "make hiring a profit center" or "provisionally hire" or do internships. Work samples done correctly demand less time from candidates than interviews and scale better than interviews. They are standardizable and iterable. What I feel like I'm reading here is someone who has been poisoned by FAANG hiring practices --- a…

I'm a little bit more on the fence with the work sample interviews having designed them and also interviewed through them. I've also done my fair share of "traditional" tech interviews, all at startups, never FAANG.

As an interviewer, I much prefer the signals generated through a work-sample interview. I'm much more confident in the hiring recommendation than I get from a 1 hour zoom session. However, if I look at teams that were built through the work-sample and zoom interviews, I'm not sure the outcomes were that noticeably better.

As an interviewee, I think I understand the frustration being on the other side has. With an in-person interview, I often have a good sense that I bombed the interview or something to improve on or replay in my head, less surprising outcomes. On the work samples it's harder to know whether you're making mistakes, or are being out-competed by someone putting in 4 times the effort to polish the solution beyond what their regular work product would be. Although I had one really good outcome where the work-sample interview really flagged the internal dysfunction of a company.

And then with both interview processes, I still think there is a really big unknown on what the false no-hire rate is, how much effort is getting wasted rejecting candidates that would actually fit the team.

So having to choose a process as an interviewer, I'm with you and would always choose a work-sample interview. On whether it should be considered the "gold standard", I'm much more hesitant, I think there are some limitations that are still hard to control for.

I do wish Starfighter/Stockfighter model had gained more traction, would've been interesting to see a recruiting company specialize in this and then seeding the interview results to multiple companies model work out.

Re: The Last Technical Interview

#56

Post by tptacek »

dpark wrote: post #53
tptacek wrote: post #51

Earlier quoted context omitted.

We don't ask or check how long candidates take. You're a professional, we give you a challenge, you can decide (up front, 30 minutes in, whatever) whether you're likely to be able to finish in the budgeted time. Maybe you can't because you've got a lunch date and don't have the contiguous block, and want to do it in chunks. Fine by us. Again, the underlying smuggled premise here is that candidates have to finish the…

You are assuming the assignment is reasonable and the candidate is lacking if they cannot complete in the expected time. And for all I know that’s entirely true for you. What I know is that I’ve seen assignments from others where the assignment scope was unreasonable for the allotted time. And for those teams, the filter becomes not so much “is this person capable” but “is this person willing to put up with our shit”…

We do assess all of those things. Again: you're a professional. We give you a work sample test. You look at it, and use your best professional judgement to decide if it is (a) reasonable and (b) doable in the budgeted time given your capabilities. If either is untrue, you don't do the work sample.

I'm having a real hard time seeing how this isn't strictly better than an interview, which, as the article (and basically everything written in the last year about interviews) points out, is basically a random function.

Re: The Last Technical Interview

#57

Post by Traubenfuchs »

kudokatz wrote: post #20
nitwit005 wrote: post #7

I gave the feedback at one Google interview that they should send Google employees through to see how many get hired. Good to see they basically tried that. The conclusion at the end that bringing someone on board is the ideal method is true I'm sure, but even that runs into the issue that employee evaluation is an even worse situation than the interview process. You can openly see some managers panic when they reali…

2 anecdotes ... 1) The worst interview I ever had (BY FAR) was at Google--disrespectful people, no respect for time, I could go on and on. And I went back to try again to get that money showered on me. Worth it in the long run. 2) Their new system for "performance management" is a hoax. Just like at all other places, it "documents" what you should do so they can fire you more easily with unspoken rules and all sorts…

I once failed all my goals agreed upon a year ago but got promoted anyways, because priorities had shifted and the director above me just really, really wanted me to continue building his reputation. (not at google)

Re: The Last Technical Interview

#58

Post by chihuahua »

The "provisional employment" idea sounds good at first, until you think about how it would actually work in practice. You have 100 applicants for 1 position. Which one do you provisionally hire?

Ah of course, we have to do a traditional interview loop to evaluate 10 candidates before we can pick one. So you do the traditional interview loop, and then you have 6 months of provisional employment.

You haven't replaced anything, you've just added another level of hassle for everyone.

Re: The Last Technical Interview

#60

Post by sgillen »

chihuahua wrote: post #58

The "provisional employment" idea sounds good at first, until you think about how it would actually work in practice. You have 100 applicants for 1 position. Which one do you provisionally hire? Ah of course, we have to do a traditional interview loop to evaluate 10 candidates before we can pick one. So you do the traditional interview loop, and then you have 6 months of provisional employment. You haven't replaced a…

I think you've just lowered the risk of a bad hire for the company, which might allow them to "take a chance" on candidates they might otherwise pass on.
Post reply on HN