I think this method is good in theory but fails in practice. Recently, I applied to three different companies that asked me to do a code sample that took from about two to eight hours each. I generally don't work for free, but I had some spare time and decided why not? I did all three and turned them in. Out of three, one company liked my sample but in the interview that followed, decided that what I was asking was too much (it's what I'm currently making actually). One company never even looked at the code and after two weeks, emailed back saying they had forgotten and decided to hire someone else. The third company was pretty responsive and I'll be going in for an in-person. A seven hour in-person interview that will be more coding and such (at least on a laptop so it's better than most places). So one out of three. I think this is a huge problem with this approach and basically, most companies are not responsible enough to follow this approach. They drop the ball constantly and wonder why they can't find good engineers when they're literally everywhere. Put into the hands of a respectable company, I would definitely prefer this in the future rather than the typical white board bullshit and trivia slinging.
The Hiring Post
161–170 of 266 posts
Re: The Hiring Post
#162My biggest issue with the work sample tests is the companies that waste your time with them. I've had 3 cases over the years where I was asked to work on something, I did it quickly, with quality and exceeded the specs and I was still turned down at that point (I confirmed that the code was good in at least 2 of the cases). As a senior technical person at this point in my career I simply have very little stomach for…
Yup. Just had this happen. I don't think I will be doing anymore coding tests.
Re: The Hiring Post
#163I have had my share of bad interview questions too. The worst part is when the interviewer starts working on their laptop, like they are so bored of this question they can't think about it any longer. Yet it is now a part of my life for 45 minutes. At the end of every trivia question I'm tempted to ask: so is this typical of what I will be doing here? Will I be implementing rand(7) given rand(5) once a week? I've als…
Perhaps they were taking notes about the interview on their laptop?
I've had people do that too, but that's noticeably different because they pay attention to you.
It's obvious when people are working because you can see them context switch back to you.
Re: The Hiring Post
#164Earlier quoted context omitted.
> But if you actually know what a linked list is, this should be simple for you to derive relatively quickly (not Googling for a day). Actually I think this is the worst kind of question to ask, because it measures: can the candidate code under pressure? Not "we need to get this done by launch/before client meeting" pressure, but right now pressure. If a company wants literal coding ninjas who can reason about comput…
OK, so make it a conversation. Have them draw on paper, or talk it out. I'm probably on the wrong side of reality since plenty of people make lots of usable stuff while using vastly suboptimal approaches. OTOH if someone doesn't known very basic stuff, it seems unlikely they're gonna Google stuff and become wise. We're not talking about anything advanced; these are basic algorithms and data structures. Some people ju…
This is an even worse idea than just asking for white-board coding, but it seems to come along every time. Not only are you putting the candidate under pressure, where thinking is difficult, now you expect them to walk you through their thought process and narrate what they're thinking, disrupting the thought process. I personally need to think things on my own without having the pressure of having to tell the interviewer what I'm thinking. Not only does it alter my thought pattern and remove a lot of the ability to think, but the extra pressure from wondering whether I'm saying the right things or taking too long to think before stopping myself so I can please the interviewer pretty much removes my ability to solve any sort of complex problem I haven't solved before. Not to mention the anger under the surface at such a daft and counter-productive approach to problem solving. This might work on TV and might be useful when collaboratively brainstorming, but in an interview, the only thing it's testing is the interviewee's ability to deal with social pressure.
Re: The Hiring Post
#165Earlier quoted context omitted.
How then do you control for time pressure?
Are you asking how do you judge if someone performs under time pressure? If so, my first answer is don't. If that is a serious part of your real world work, you are doing it wrong. The less flippant answer is, you give them a deadline but it is both A) flexible in the face of their "real life" (ie if they tell you they are going on vacation next week deal with it) and B) it deals with realistic time horizons (ie a we…
Sorry my post was not complete enough - what I meant was that if you have two equally good candidates, but one happens to have experience in Rails, that person is going to perform the coding test more quickly than the other candidate i.e. get a more complete / correct solution in a given amount of time. So I hope in this situation there are no time limits?
Re: The Hiring Post
#166I think this method is good in theory but fails in practice. Recently, I applied to three different companies that asked me to do a code sample that took from about two to eight hours each. I generally don't work for free, but I had some spare time and decided why not? I did all three and turned them in. Out of three, one company liked my sample but in the interview that followed, decided that what I was asking was t…
What you've just described sounds like the exact opposite of a rigorous, objective, work sample based hiring approach. Crappy companies have crappy hiring processes. It sucks to deal with that.
When we started doing work sample hiring, we put a process in place to handle many applicants, make sure we were responsive, and work people through the funnel as efficiently as we could. I don't think it's possible to take advantage of it any other way.
We got complimented several times on how reasonable the process was (by people we turned down!). This both made me feel good, and caused me existential "the world is broken" sadness. It's not a high bar to be decent to people.
Re: The Hiring Post
#167We went through the phase when we gave candidates a problem and let them work on it remotely. It was in an embedded C shop that did a lot of kernel work. Basically, we'd give a short programming task (say, to write an intrusive AVL tree container in C) and 24 hours. Guess what? HALF of candidates cheated. Meaning that when the got called for an in-person interview, they stumbled to explain how "their" code worked. To…
Sounds like a rousing success in terms of qualifying/disqualifying candidates.
Re: The Hiring Post
#168Earlier quoted context omitted.
To me, recruiting is two problems: outreach and qualification. You're asking about outreach. How we did outreach is a whole 'nother blog post, and one I'm bound to write soon. What surprised me about recruiting was how much more important qualification is than outreach. Without good qualification, it almost doesn't matter how good your outreach is, because you're filtering for the same highly visible easily accessibl…
Maybe I wasn't clear with my question. Say the company indicates it is hiring and gets 100 interested people. How do you filter them down to serious applicants without looking at resumes? I would argue that isn't so much outreach as it is qualification, as the people are already interested. Maybe my premise of 100 candidates is incorrect based on how you did outreach?
Serious is "testable", in the sense that you can ask them a specific question about the job to see if you get a response. We warm people up on the phone when they're putting special attention into our positions, and give everyone the opportunity to answer work samples (whether they seem serious or not). The point of warming them up is to balance out the effort — we're asking people to spend a few hours on this stuff, it's fair to give them time.
Re: The Hiring Post
#169I think this method is good in theory but fails in practice. Recently, I applied to three different companies that asked me to do a code sample that took from about two to eight hours each. I generally don't work for free, but I had some spare time and decided why not? I did all three and turned them in. Out of three, one company liked my sample but in the interview that followed, decided that what I was asking was t…
What you've just described sounds like the exact opposite of a rigorous, objective, work sample based hiring approach. Crappy companies have crappy hiring processes. It sucks to deal with that. When we started doing work sample hiring, we put a process in place to handle many applicants, make sure we were responsive, and work people through the funnel as efficiently as we could. I don't think it's possible to take ad…
No, it's not a high bar to be decent to people, yet many companies (and people) still aren't. The problem then is that prospective employees have no way of knowing that a company is decent versus another company that is not. Will I do another work sample for another company? I'm thinking likely not, but I haven't ruled out the possibility. Note though, that the company must offer something of great value for me to even consider it. In my case, that's remote work. I would never consider it for a non-remote company. Period.
Still, a company could have a great, responsive process in place. But my experience, and that of others, is that it's not worth my time and effort to do work for free. Let's be clear: this is what is being asked of the candidates. It therefore starts to undermine any company using such a process even though I don't think it's an inherently bad process.
Re: The Hiring Post
#170Author. I can talk in a pretty good amount of detail about how exactly our process worked, if anyone has any questions. And, to head off a concern a reviewer gave me: from 1997-2005, I was a full-time software developer; I shipped shrink-wrap boxed software on Windows and Unix in the 1990s, then appliances deployed at tier 1 ISPs. I'm a "software person" more than a "security person".
Amazing text, last month I have screwed up at an interview just because I had no knowledge about what they were asking. After the interview I google about it for 2 minutes and then I was able to quickly answer that question.