Live data from Hacker News

Ask HN: Measuring the long-term benefit of interview code tests?

news.ycombinator.com

11–20 of 76 posts

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#11
post #10

We don't use coding tests in this way. We use coding tests as a screening process to ensure the candidate is in the correct ballpark. If we are recruiting a senior, we would expect them to easily complete basic technical tests. If they are more junior we might use them only as an indicator of their ability. I don't particularly expect a strong correlation between how well they did in the tests and their long-term abi…

Do you do it for the interview process as a whole?

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#12

For developers, coding tests that include deployment / infrastructure components (i.e., deploy your solution to a cloud container, or, build and compile your solution for desktop platform testing) are uniformly consistent with long-term impact / success. Problem solving at the algorithmic layer may be inversely correlated to success, if a candidate lacks a production skill set. Unless one's focus is research and deve…

Nonsense, on several levels:

1) If you join a company like Google, there's no way to learn their production tools in advance.

2) Even if you join a company that uses today's cloud or Open Source tools de jour, that can and will change quickly.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#16
I've never worked for a company that did (18 years in the industry this year). Of the 8 companies I've worked for, only one had interviewing figured out, and they didn't track or measure metrics on coding tests, challenges, etc. They did allow the challenges to evolve and they were tailored to the position that the tests/challenges were for.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#17

Of course not. However, we do hire some contractors essentially without an interview, and it is fairly apparent that's a bad idea.

We tried that at one company I worked for and it worked well enough. Our contract with the consulting firm said if we dumped the contractor within 90 days we didn't pay a cent for any of their time. This resulted in the consulting firm only sending us candidates that had good reviews in prior engagements. And good reviews from prior engagements strongly correlates with good reviews in future engagements.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#18
The company would have to be fairly large (>100 employees) and long-lived (>10 years) to generate an amount of data with any hope of statistical significance. Employee "success" depends on many factors, and an employee who can seem to be a failure in the short-term may end up becoming very successful (or vice-versa), simply because of external circumstances---the nature of the projects, the clients, colleagues, etc.

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#19

Of course not. However, we do hire some contractors essentially without an interview, and it is fairly apparent that's a bad idea.

We tried that at one company I worked for and it worked well enough. Our contract with the consulting firm said if we dumped the contractor within 90 days we didn't pay a cent for any of their time. This resulted in the consulting firm only sending us candidates that had good reviews in prior engagements. And good reviews from prior engagements strongly correlates with good reviews in future engagements.

Can you tell me which consulting company was that. We have a few contract positions where we can use consultants .

Re: Ask HN: Measuring the long-term benefit of interview code tests?

#20
The only way this could even conceivably be done in a scientifically valid way is randomized controlled trials, which would mean not giving the same interviews to all candidates, which is only possible if hiring at a large enough scale as to even be able to sample meaningfully from multiple "interview type" groups, and it would of course require it to be legal to give different interviews at random, which I'm not sure is true. I guess as long as it's actually random, you're not discriminating against any specific group, but it isn't exactly fair and you risk killing goodwill of your employees when they realize you're running experiments on them.

Of course, it's really not possible at all to do this at the level of rigor expected of, say, clinical trials. Each new hire will know what type of interview you put them through, and there is no reliable way you can prevent them from telling others.

Post reply on HN