Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Measuring the long-term benefit of interview code tests?

news.ycombinator.com

Post reply on HN
Print view View original

31–40 of 76 posts

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#31

Post by alberth »

kasey_junk wrote: post #29
alberth wrote: post #24

Earlier quoted context omitted.

Very interesting. Question: How did you measure the candidate once hired? What factors were indicative of a "good" hire vs. a "bad" hire?

We compared their performance review scores. I was always leery of this given how fraught performance reviews are, but that’s how the company judged employee ‘worth’ so it made sense to align there.

And just to pull on this thread a bit more ... what factored into your employer (or anyone reading my post) rating of an engineers performance?

Is it largely based on soft skills assessment. Or somehow code quality is being judge?

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#32

Post by leeoniya »

kasey_junk wrote: post #8

I don't know if my current company does, but when I first implemented them for a company I worked for ~15 years ago we definitely did. At that company (which was a ~200 engineer, privately held, software company) we found a few things: - in person tests were less predictive than take home tests. - tests that did not provide automated test cases as examples were less predictive than those that did. - there was virtual…

> there was virtually no predictive power to 'secret test cases' that we ran without providing to the candidate.

this brings back some unpleasant memories of a take-home i got from a FAANG.

basically i was given a loose spec to implement, with no real data or test cases (and was told that none would be provided when i asked). after submitting my work i received a terse rejection with 0 constructive feedback for my 6hrs of work. uncool.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#33

Post by nefitty »

kasey_junk wrote: post #8

I don't know if my current company does, but when I first implemented them for a company I worked for ~15 years ago we definitely did. At that company (which was a ~200 engineer, privately held, software company) we found a few things: - in person tests were less predictive than take home tests. - tests that did not provide automated test cases as examples were less predictive than those that did. - there was virtual…

The clear requirements of take home tests make them my favorite. They allow me to express how I work: get a list of reqs, walk away and think about them, make some decisions on directions, then let the code lead me.

I strike the style required. I capitalize on opportunities to make decisions I can discuss. "I used tape instead of jest because this example product will be distributed to many developers. The reduced API-surface area keeps us focused on the how's not the what's."

I tone that down if the role seems more rote-work like, at which point I try to highlight my ability to solve problems and learn quickly. For example, a comment above some network call: "// I was getting a cors error and found out I can run my own proxy for this"

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#34

Post by rednerrus »

meheleventyone wrote: post #11
lbriner wrote: post #10

We don't use coding tests in this way. We use coding tests as a screening process to ensure the candidate is in the correct ballpark. If we are recruiting a senior, we would expect them to easily complete basic technical tests. If they are more junior we might use them only as an indicator of their ability. I don't particularly expect a strong correlation between how well they did in the tests and their long-term abi…

Do you do it for the interview process as a whole?

We do this. We have a scale of how in-depth we expect people to get based on their level. We ask candidates questions about specific projects they've worked on and we ask them to be more specific until we feel we have a good understanding of how deep and wide their understanding is. We dig deeper for more senior folks. It seems to work pretty well.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#35

Post by handrous »

nefitty wrote: post #33
kasey_junk wrote: post #8

I don't know if my current company does, but when I first implemented them for a company I worked for ~15 years ago we definitely did. At that company (which was a ~200 engineer, privately held, software company) we found a few things: - in person tests were less predictive than take home tests. - tests that did not provide automated test cases as examples were less predictive than those that did. - there was virtual…

The clear requirements of take home tests make them my favorite. They allow me to express how I work: get a list of reqs, walk away and think about them, make some decisions on directions, then let the code lead me. I strike the style required. I capitalize on opportunities to make decisions I can discuss. "I used tape instead of jest because this example product will be distributed to many developers. The reduced AP…

Trouble is, unless more of the industry starts doing them so they're unavoidable, I'm going to skip companies that put these anywhere other than at the tail end of their process.

I'm not putting in half a day of work for zero pay to help you with your first-pass weed-out phase before we bother to make sure we align otherwise and this looks like a good fit. Thanks, bye, next (employer) candidate.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#36

Post by Aeolun »

We don’t do coding tests at all. We do one 30-60m interview that covers some general tech questions and motivation.

So far we’ve hired 7 decent and 3 great people. No truly bad people have made it through that pipeline yet.

I can’t say anything about why, and I’d be prejudiced in any case.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#38

Post by kasey_junk »

alberth wrote: post #31
kasey_junk wrote: post #29

Earlier quoted context omitted.

We compared their performance review scores. I was always leery of this given how fraught performance reviews are, but that’s how the company judged employee ‘worth’ so it made sense to align there.

And just to pull on this thread a bit more ... what factored into your employer (or anyone reading my post) rating of an engineers performance? Is it largely based on soft skills assessment. Or somehow code quality is being judge?

I honestly dont remember. It’s been a long time and I wasn’t privy to the assessments just the scores (and they were anonymized before I saw them). I do remember them being several 5 point scores.

It wouldn’t surprise me if they were just manager assessments.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#39

Post by 999900000999 »

boldslogan wrote: post #2

and maybe a follow up question (to measure the false negatives) Do you check the applicants who were denied based on their test and see where they ended up working at. E.g. you are a mid tier start up who rejects someone who ends up working at amazon as a high level engineer – do you mark that a failure?

Let’s think about this for a second, if I apply to be a mid-level engineer at Billy Bob’s software development firm, but I’m capable of getting a job as a senior engineer at Amazon. Odds are I’d only stay at Billy Bob’s software up until I’m able to get my Amazon job, considering on boarding a software engineer can easily take six months, that means you only get six months or so of actual work out of this person. If that, they might just work at Billy Bob’s for three months until they get their FAANG offer and then just leave Billy Bob’s off the résumé

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#40

Post by waynesonfire »

handrous wrote: post #35
nefitty wrote: post #33

Earlier quoted context omitted.

The clear requirements of take home tests make them my favorite. They allow me to express how I work: get a list of reqs, walk away and think about them, make some decisions on directions, then let the code lead me. I strike the style required. I capitalize on opportunities to make decisions I can discuss. "I used tape instead of jest because this example product will be distributed to many developers. The reduced AP…

Trouble is, unless more of the industry starts doing them so they're unavoidable, I'm going to skip companies that put these anywhere other than at the tail end of their process. I'm not putting in half a day of work for zero pay to help you with your first-pass weed-out phase before we bother to make sure we align otherwise and this looks like a good fit. Thanks, bye, next (employer) candidate.

sweet, don't let the door hit you. More opportunities for me.
Post reply on HN