Ask HN: Measuring the long-term benefit of interview code tests? • Hacker News Board

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#71

Post by roland35 » Sat, Nov 06, 2021, 2:32 PM UTC

kovac wrote: post #68

leeoniya wrote: post #32

Earlier quoted context omitted.

> there was virtually no predictive power to 'secret test cases' that we ran without providing to the candidate. this brings back some unpleasant memories of a take-home i got from a FAANG. basically i was given a loose spec to implement, with no real data or test cases (and was told that none would be provided when i asked). after submitting my work i received a terse rejection with 0 constructive feedback for my 6h…

In one interview, I was given a timed hacker rank problem with a screen share with 2 interviewers. The sample tests passed and the real tests passed except for 2 (from what I remember) that timed out on large data sets. Before the tests were run, I already highlighted the part of the code that's the bottleneck and asked if I could copy the code to Visual Studio (the test was in C#) because the standard lib has a data…

I always feel like that type of coding interview is a sort of engineering hazing. I know I am often consulting documentation, especially when working in a new problem-space or less familiar programming language!

I always try to give candidates the benefit of doubt with silly things like syntax or whatever since it's not like I'm interviewing for a live coding performer!

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#72

Post by roland35 » Sat, Nov 06, 2021, 2:35 PM UTC

kasey_junk wrote: post #55

errcorrectcode wrote: post #51

Earlier quoted context omitted.

I think we should use interviews for basic screening purposes only and skip determining "who is awesome from just signals". Instead, shift to a flexible hours paid trial period where potential colleagues get a better assessment. Measure by doing and interacting around doing, not by guessing, hazing, trivia, interrogating, or whiteboard hand-waving.

The issue with paid trials are 2 fold: If you are talking about short term trials many devs are bound by anti-moonlighting employment agreements that either outright bar working for someone else or require notification. For long term trials you severely limit your hiring pool because that is effect temp-to-hire which many devs simply will not do.

This would maybe work if every employer did it and it was easy to pick up a new trial quickly, but the reality is that the time from application to hire can be weeks if not months at most companies!

No way I could risk having to find another job if the trial went poorly.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#73

Post by impute » Sat, Nov 06, 2021, 2:35 PM UTC

leeoniya wrote: post #32

kasey_junk wrote: post #8

I don't know if my current company does, but when I first implemented them for a company I worked for ~15 years ago we definitely did. At that company (which was a ~200 engineer, privately held, software company) we found a few things: - in person tests were less predictive than take home tests. - tests that did not provide automated test cases as examples were less predictive than those that did. - there was virtual…

> there was virtually no predictive power to 'secret test cases' that we ran without providing to the candidate. this brings back some unpleasant memories of a take-home i got from a FAANG. basically i was given a loose spec to implement, with no real data or test cases (and was told that none would be provided when i asked). after submitting my work i received a terse rejection with 0 constructive feedback for my 6h…

Same experience. This is why I will no longer do take home tests that take more than 90 minutes or look like they'll take more than 90 minutes (even if the company misjudges it).

The only exception I've made is if the company pays for the time.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#74

Post by singron » Sat, Nov 06, 2021, 3:05 PM UTC

908B64B197 wrote: post #54

kasey_junk wrote: post #8

I don't know if my current company does, but when I first implemented them for a company I worked for ~15 years ago we definitely did. At that company (which was a ~200 engineer, privately held, software company) we found a few things: - in person tests were less predictive than take home tests. - tests that did not provide automated test cases as examples were less predictive than those that did. - there was virtual…

The problem with keeping these stats is that it only tracks engineers that were hired. I don't think coding interviews are a good predictor of performance, and that's not why I use them. The point of a coding interview is to eliminate, as fast as possible, people who simply can't code. I'm being completely serious here. They can even have a CS degree (or will claim to but if you look closely they were in an easier pr…

We used the same takehome for years, and eventually there were a few solutions online that were easy to find. But for some reason, they all sucked, so we never had to worry about unqualified candidates copying them.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#75

Post by kasey_junk » Sun, Nov 07, 2021, 3:09 AM UTC

impute wrote: post #73

leeoniya wrote: post #32

Earlier quoted context omitted.

> there was virtually no predictive power to 'secret test cases' that we ran without providing to the candidate. this brings back some unpleasant memories of a take-home i got from a FAANG. basically i was given a loose spec to implement, with no real data or test cases (and was told that none would be provided when i asked). after submitting my work i received a terse rejection with 0 constructive feedback for my 6h…

Same experience. This is why I will no longer do take home tests that take more than 90 minutes or look like they'll take more than 90 minutes (even if the company misjudges it). The only exception I've made is if the company pays for the time.

Fwiw that job we had an explicit goal of 60 minutes or less and tested that against engineers we’d already hired.

I’ve heard guidance that said up to 4 hours was a fine. That might have been true back then before employers abused the system and made code tests just another hoop, not a replacement for, interviewing.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#76

Post by gcheong » Mon, Nov 08, 2021, 12:47 AM UTC

908B64B197 wrote: post #54

kasey_junk wrote: post #8

I don't know if my current company does, but when I first implemented them for a company I worked for ~15 years ago we definitely did. At that company (which was a ~200 engineer, privately held, software company) we found a few things: - in person tests were less predictive than take home tests. - tests that did not provide automated test cases as examples were less predictive than those that did. - there was virtual…

The problem with keeping these stats is that it only tracks engineers that were hired. I don't think coding interviews are a good predictor of performance, and that's not why I use them. The point of a coding interview is to eliminate, as fast as possible, people who simply can't code. I'm being completely serious here. They can even have a CS degree (or will claim to but if you look closely they were in an easier pr…

That was the purpose of the original fizzbuzz but for whatever reason it seems to have morphed into “spend all your spare time on leetcode so that you can answer whatever arbitrary problem is thrown at you” and they have the audacity to call that a meritocracy.