Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Measuring the long-term benefit of interview code tests?

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 76 posts

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#51

Post by errcorrectcode »

kasey_junk wrote: post #8

I don't know if my current company does, but when I first implemented them for a company I worked for ~15 years ago we definitely did. At that company (which was a ~200 engineer, privately held, software company) we found a few things: - in person tests were less predictive than take home tests. - tests that did not provide automated test cases as examples were less predictive than those that did. - there was virtual…

I think we should use interviews for basic screening purposes only and skip determining "who is awesome from just signals". Instead, shift to a flexible hours paid trial period where potential colleagues get a better assessment. Measure by doing and interacting around doing, not by guessing, hazing, trivia, interrogating, or whiteboard hand-waving.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#52

Post by psadri »

I'd like to point out that success in a role depends on more factors than the technical interview.

I have found that investing the time to correctly onboard new team members makes a huge difference. Correctly onboard an average/good hire and they go on to produce solid output and often thrive. On the other hand, you could have a great new hire but because of no/poor onboarding they "sink" instead of swim.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#53

Post by kspacewalk2 »

sosborn wrote: post #48
kspacewalk2 wrote: post #5

Earlier quoted context omitted.

On the other hand, "they would not have been a good fit" sounds suspiciously like a blanket, non-falsifiable denial of failure. I other words, bullshit.

Is it bullshit if it is true? Hiring the wrong fit can bring a team down.

It's not bullshit if it's true. It can be true, or it can be bullshit. Which makes it such an awesome bullshit excuse.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#54

Post by 908B64B197 »

kasey_junk wrote: post #8

I don't know if my current company does, but when I first implemented them for a company I worked for ~15 years ago we definitely did. At that company (which was a ~200 engineer, privately held, software company) we found a few things: - in person tests were less predictive than take home tests. - tests that did not provide automated test cases as examples were less predictive than those that did. - there was virtual…

The problem with keeping these stats is that it only tracks engineers that were hired. I don't think coding interviews are a good predictor of performance, and that's not why I use them.

The point of a coding interview is to eliminate, as fast as possible, people who simply can't code. I'm being completely serious here. They can even have a CS degree (or will claim to but if you look closely they were in an easier program to get into and took CS electives) but cannot write a simple program on the board in an hour.

It's also why I don't like take-homes. First it's trivial to cheat (I don't mean lookup stuff online, just flat out have someone else do the work) and because of that the final stage would still have to be in-person whiteboard (or pair programming over Slack but still have an engineer spend 40+ minutes with the candidate).

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#55

Post by kasey_junk »

errcorrectcode wrote: post #51
kasey_junk wrote: post #8

I don't know if my current company does, but when I first implemented them for a company I worked for ~15 years ago we definitely did. At that company (which was a ~200 engineer, privately held, software company) we found a few things: - in person tests were less predictive than take home tests. - tests that did not provide automated test cases as examples were less predictive than those that did. - there was virtual…

I think we should use interviews for basic screening purposes only and skip determining "who is awesome from just signals". Instead, shift to a flexible hours paid trial period where potential colleagues get a better assessment. Measure by doing and interacting around doing, not by guessing, hazing, trivia, interrogating, or whiteboard hand-waving.

The issue with paid trials are 2 fold:

If you are talking about short term trials many devs are bound by anti-moonlighting employment agreements that either outright bar working for someone else or require notification.

For long term trials you severely limit your hiring pool because that is effect temp-to-hire which many devs simply will not do.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#56

Post by drBonkers »

leeoniya wrote: post #32
kasey_junk wrote: post #8

I don't know if my current company does, but when I first implemented them for a company I worked for ~15 years ago we definitely did. At that company (which was a ~200 engineer, privately held, software company) we found a few things: - in person tests were less predictive than take home tests. - tests that did not provide automated test cases as examples were less predictive than those that did. - there was virtual…

> there was virtually no predictive power to 'secret test cases' that we ran without providing to the candidate. this brings back some unpleasant memories of a take-home i got from a FAANG. basically i was given a loose spec to implement, with no real data or test cases (and was told that none would be provided when i asked). after submitting my work i received a terse rejection with 0 constructive feedback for my 6h…

I've recently encountered similar assessments. I asked for feedback or the test cases but got none. What do you think the best option is to learn from the projects?

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#57

Post by leeoniya »

drBonkers wrote: post #56
leeoniya wrote: post #32

Earlier quoted context omitted.

> there was virtually no predictive power to 'secret test cases' that we ran without providing to the candidate. this brings back some unpleasant memories of a take-home i got from a FAANG. basically i was given a loose spec to implement, with no real data or test cases (and was told that none would be provided when i asked). after submitting my work i received a terse rejection with 0 constructive feedback for my 6h…

I've recently encountered similar assessments. I asked for feedback or the test cases but got none. What do you think the best option is to learn from the projects?

post it on stack overflow or reddit for feedback :D

typically they tell you not to post your solutions publicly, though i don't know why you'd be inclined to respect their wishes after such an experience unless you're dying to work there in the future; the main thing i learned from the project is that i didn't.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#58

Post by daviddever23box »

necovek wrote: post #21
daviddever23box wrote: post #3

For developers, coding tests that include deployment / infrastructure components (i.e., deploy your solution to a cloud container, or, build and compile your solution for desktop platform testing) are uniformly consistent with long-term impact / success. Problem solving at the algorithmic layer may be inversely correlated to success, if a candidate lacks a production skill set. Unless one's focus is research and deve…

With the original question being about measuring correlation between interview performance and actual long-term performance on a job, I'd love to see the numbers you are basing your opinion on regarding testing for "production skills" and long-term job performance.

Go ahead, show me how team interaction and declared responsibility can be measured unambiguously.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#59

Post by necovek »

daviddever23box wrote: post #58
necovek wrote: post #21

Earlier quoted context omitted.

With the original question being about measuring correlation between interview performance and actual long-term performance on a job, I'd love to see the numbers you are basing your opinion on regarding testing for "production skills" and long-term job performance.

Go ahead, show me how team interaction and declared responsibility can be measured unambiguously.

Here's a couple simple examples:

- Boolean yes/no qualification by a manager at 2 years of tenure compared to a boolean yes/no qualification upon performing the interview task (yes, they showed skills that relate to deployment)

- Performance review ratings over time (I would assume those cover "team interaction" and "declared reponsibility")

- Length of tenure ("lasted more than 6 months", ..., "lasted more than 5 years", "quit themselves"...).

...

None of these are perfect, but they are pretty unambiguous even if sometimes subjective.

As long as you measure while fully understanding that you're unlikely to achieve statistical significance soon, you can still measure stuff! And while you should not take trends that show up as gospel, they could still influence your interviewing.

Or, you could simply decide not to measure anything since you'll never get perfect results. But at least don't put your opinion out as fact, at least not in a thread about measuring stuff.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#60

Post by killingtime74 »

errcorrectcode wrote: post #51
kasey_junk wrote: post #8

I don't know if my current company does, but when I first implemented them for a company I worked for ~15 years ago we definitely did. At that company (which was a ~200 engineer, privately held, software company) we found a few things: - in person tests were less predictive than take home tests. - tests that did not provide automated test cases as examples were less predictive than those that did. - there was virtual…

I think we should use interviews for basic screening purposes only and skip determining "who is awesome from just signals". Instead, shift to a flexible hours paid trial period where potential colleagues get a better assessment. Measure by doing and interacting around doing, not by guessing, hazing, trivia, interrogating, or whiteboard hand-waving.

Why would I go for a job with a trail period when there are plenty others that give me certainty immediately?
Post reply on HN