Why we don't hire programmers based on puzzles and tricks

I've done my fair share of interviews. I still don't like them. I've narrowed it down to basically two things:

1. There is a lot of highly subjective, mystical advice about what "works," in a technical interview and what doesn't. Big problems, small problems, trivia, puzzles, white-boards, no white-boards... hardly any of it is thoroughly studied and built on solid theory. It's all conjecture and personal bias.

2. How the candidate is feeling and where their head is at on the day of the interview matters. I can go on about cache alignments, bus errors, segmentation faults, C++'s lack of order in evaluating function arguments, why template class declarations must exist in the same compilation unit as the definition, why CLOS is amazing and the computation model of recursion is brilliant (or at least why I think so). Yet on a bad day I can (and have) botched perfectly trivial, benign problems.

However at the end of the day you need some sort of process. I just think that the current practices are not good enough. It seems to me that companies are probably turning away perfectly fine candidates without realizing it. There's pros and cons to every approach I suppose. But I still think trivial problems are dumb.

Take the process of advertising a position and moving candidates through the hiring process - the tools there are almost universally crappy and haven't made a lot of progress in the last decade.

Also, as far as I can tell, very few companies attempt to record any kind of data about how candidates are hired and how they subsequently performed on the job (even tracking job performance is a tough problem) - and it certainly doesn't exist industry-wide.

There is a ton of money in this area - why hasn't there been much progress? Is it because it is an unsexy problem, or is it because it is genuinely impossible?

Everyone has performance anxiety at times - whether rare, mild, common, or occasionally overwhelming it's something everyone deals with. Something like public speaking to a room of strangers or flirting with a super model really doesn't share many physiological triggers with diagnosing code you wrote while people you're friends with look on anxiously.

Assuming that because something's easy for you (and most people) that anyone who has trouble with it is likely to be broken in lots of ways isn't too safe a bet.

If someone gets overwhelmed by vertigo on a skyscraper does that mean you won't be able to trust them to drive in rush hour traffic? If a guy pisses his pants when he gets a gun pulled on him does that mean he can't be trusted to be an airline pilot?

Those are exaggerated to make the point, but interviewing is an art. You have great latitude in terms of making people comfortable or uncomfortable depending on your goals. With enough effort I'd be willing to bet you could engineer an interview that would send almost anyone over the edge.

So now that it's just a case of degrees, all you have to do is decide whether you're hiring someone that needs to be able to solve programming puzzles in front of a stranger that's actively critiquing them.

Unless you are specifically hiring people for a death march kind of project (which I find unwise anyway), targeting people good at high-stress situations doesn't seem very productive.

If your programming projects are a series of high-stress, high-adrenaline runs, either you work for NASA or you're doing it wrong.

You're not asking about syntax with this. It's a problem solving question and at the end of the day, the only real metric here is not whether a dev could code this up, but whether he can code it up quickly, under time constraints, while being stared at, under pressure, with rent or a mortgage on the line, and maybe kids to feed.

I am hard pressed to think of even one real life dev task I had in my entire career where I had to solve a problem under those circumstances. For most devs, we take ownership of the problem and solve it at our desk under reasonable time frames without the spotlight. For bigger problems, we might think about it in the car on the way home. We might sleep on it over the weekend.

That doesn't apply to reversing a string, of course. But you have to consider that, depending on the dev's background and skills, he may be spending time in jQuery, maybe Spring and Hibernate (for Java guys), maybe even some CSS. Maybe he's a GWT guy? Maybe he was assigned to a team dedicated to (God forbid) EJBs. In other words, he may be multifaceted and spends more time working with an API than "just" raw language coding. The language is just a means for working with the API.

And the one thing you want to measure is not something he does daily. Now, if he can't use a for loop, feel free to mock him. But reversing a string? That's clearly a puzzle.

EDIT: I can't get this part out of my mind: "Is incredible how badly the test goes with the people! I interview for myself or others more than 100 people (of any background, including university, tech schols...) and I think only 10/12 people do it correctly - barely-"

So you continue to use that question? Despite the results? So that when you do hire a guy that passes, he won't ever do this on the job because you know you damn well he should use the reverse() method? Because you won't pay him to code it raw and waste time and money when there is a 1.5 second solution to this? That's sadistic. :-)