41–50 of 76 posts
We don’t do coding tests at all. We do one 30-60m interview that covers some general tech questions and motivation. So far we’ve hired 7 decent and 3 great people. No truly bad people have made it through that pipeline yet. I can’t say anything about why, and I’d be prejudiced in any case.
We don’t do coding tests at all. We do one 30-60m interview that covers some general tech questions and motivation. So far we’ve hired 7 decent and 3 great people. No truly bad people have made it through that pipeline yet. I can’t say anything about why, and I’d be prejudiced in any case.
Earlier quoted context omitted.
The clear requirements of take home tests make them my favorite. They allow me to express how I work: get a list of reqs, walk away and think about them, make some decisions on directions, then let the code lead me. I strike the style required. I capitalize on opportunities to make decisions I can discuss. "I used tape instead of jest because this example product will be distributed to many developers. The reduced AP…
Trouble is, unless more of the industry starts doing them so they're unavoidable, I'm going to skip companies that put these anywhere other than at the tail end of their process. I'm not putting in half a day of work for zero pay to help you with your first-pass weed-out phase before we bother to make sure we align otherwise and this looks like a good fit. Thanks, bye, next (employer) candidate.
I don't know if my current company does, but when I first implemented them for a company I worked for ~15 years ago we definitely did. At that company (which was a ~200 engineer, privately held, software company) we found a few things: - in person tests were less predictive than take home tests. - tests that did not provide automated test cases as examples were less predictive than those that did. - there was virtual…
Did you test predictive power from individual interviewers? At a company I worked at previously we did, and this was the by far the best overall predictor: some interviewers just did a much better job at identifying those likely to succeed than others. Which can explain another reason why you didn't see much predictive power if you looked across those other items over all interviewers - the variance between interviewers essentially "swamps" any smaller differences between those interview techniques.
Note this didn't surprise me that much, as you see this dynamic in lots of other "person-to-person" endeavors. For example, when looking at whether one type of psychotherapy intervention is better than another, most of the data that I've seen shows that by far the most important factor is the skill and "match" between therapist and client, far more important than any individual modality.
The submitted question seems to just brush over this aspect, but so far when I've tried to evaluate interviewing techniques that has been the primary obstacle; people just can't agree on what success means once employed, so anything that tries to correlate interviewing to that will be an equal amount of junk.
I interviewed hundreds of technical people in my career, across dev, test, and ops skill sets. I saw limited correlation between tests and aptitude. If you talk to someone about a project they've done, you know pretty quickly:
1) Can they communicate technical ideas? 2) Can I develop a rapport with this person and work together? 3) Do they understand what they built? Can they talk about tradeoffs they made? Did they learn anything from the experience?
A fizz buzz test isn't a terrible idea, but you also have to have an interviewer that understands how to administer it within the wider context of the interview. If the interviewer themselves doesn't understand it, they aren't qualified to actually administer it.
The company would have to be fairly large (>100 employees) and long-lived (>10 years) to generate an amount of data with any hope of statistical significance. Employee "success" depends on many factors, and an employee who can seem to be a failure in the short-term may end up becoming very successful (or vice-versa), simply because of external circumstances---the nature of the projects, the clients, colleagues, etc.
This seems like one of those occasions where improving your reliability by just a few percent (even if far from statistical significance) can massively reduce costs in the long run. (Maybe Kahneman even used interviewing as a concrete example of this in his latest book?)
Earlier quoted context omitted.
I'd be careful to call that out as a negative; if the culture fit wasn't right, and the candidate would have been a net negative to the team, it shouldn't matter where they end up next, unless (of course) the candidate that was actually hired ends up being an even worse fit (ergo the need to fix your hiring process).
On the other hand, "they would not have been a good fit" sounds suspiciously like a blanket, non-falsifiable denial of failure. I other words, bullshit.
I don't know if my current company does, but when I first implemented them for a company I worked for ~15 years ago we definitely did. At that company (which was a ~200 engineer, privately held, software company) we found a few things: - in person tests were less predictive than take home tests. - tests that did not provide automated test cases as examples were less predictive than those that did. - there was virtual…
> no other part of the interview pipeline was predictive at all. Not whiteboarding, not presenting, not personality interviews, not culture fit testing, not credentials, or where experience came from, nothing Did you test predictive power from individual interviewers ? At a company I worked at previously we did, and this was the by far the best overall predictor: some interviewers just did a much better job at identi…
Again, we didn’t have enough data points for real statistical validity so it could be that, but I became convinced that it didn’t matter who was interviewing or the format of the interview. Some candidates are good at interviewing and some aren’t but that didn’t hold to the job.
We don't use coding tests in this way. We use coding tests as a screening process to ensure the candidate is in the correct ballpark. If we are recruiting a senior, we would expect them to easily complete basic technical tests. If they are more junior we might use them only as an indicator of their ability. I don't particularly expect a strong correlation between how well they did in the tests and their long-term abi…
Do you do it for the interview process as a whole?
There is only time I was still unsure and didn't want to waste the candidates time so rather than telling him sorry, I set him a paid coding test to develop a microservice in order to judge his style, how long he took, what questions he asked etc. I didn't think the result was good enough but because we paid him, we parted on good terms and he had some good feedback.