Why we don't hire programmers based on puzzles and tricks

The day was broken up into sessions of about 1 hour each (maybe a bit more). In each session it was me and 3-4 other people, but the people rotated, so I got to meet lots of engineers, a PM or two, the hiring manager, etc., and they all got to meet me. A few people showed up more than once.

The first session was basically an introduction. I think it was shorter than the others. No quiz-style questions, but stuff about my background, etc., and opportunities for me to ask questions also.

Next I did a presentation at the whiteboard describing some project I'd worked on. They had told me this would be a part of the day, so I could have prepared (though I didn't). I talked about my recent startup, describing some of the data model and machine learning tricks I'd developed. They asked lots of questions, some to clarify, others about my reasoning, how it would scale, if I'd considered technology X, etc.

In the next session was me at the whiteboard again, but here they described a new feature they'd like to add to their site, and my job was to sketch out a rough solution. So partially this was about my ability to ask clarification questions, and also about my big-picture design skills. Again they asked lots of questions, and once we even got into a specific SQL query (where I opted to use an EXISTS with a correlated sub-query). So this session seemed a lot like the first, but more impromptu on my part and probably more practiced on theirs.

Then they took the whole team out to lunch at a nice Thai restaurant (~15 devs plus the director of engineering and the PM). This part was relaxing and fun. Of course this was still part of the interview, and wouldn't be relaxing for everyone, but still it was a good way to feel out personality fit for both sides. Also there were too many people for me to be the focus of attention, so that was a nice break.

Finally there was one afternoon session where they gave me a laptop with a toy Rails app, and had me add a feature (change a relationship from one-to-one to one-to-many, or something like that). There were two devs watching me code, and I had to do the whole MVC thing: write a migration, tweak the controller, edit the view. I got to talk through the changes as I did them, so they knew my reasoning. It made me smile to slip in a few fancy vim moves; I don't know if they noticed. But it made me realize this was also a test about some basic mechanics, too. They let me choose my editor/IDE, so it was a chance for them to watch whether I knew my tools.

Then there was a final session for follow-up questions etc. This was fairly short I believe.

All along the way I had lots of opportunities to ask questions of my own, which I appreciated.

There were no puzzles, no trivia quizzes. Personally I'd add a FizzBuzz test to the initial phone screen (like Han Solo "They hardly asked me any questions."), but otherwise I plan to completely rip off this template the next time I hire someone myself. Maybe it can help some of you!

I'm sure it is useful in some situations, particularly in big companies with a broken hiring process. I imagine 37Signals don't find it useful as they have a more effective screening process, and people coming to interview simply couldn't possibly be that bad as they're referred or they've seen some code they wrote.

If you are really faced with candidates who are incompetent (i.e. HR has filtered candidates for you based on CVs or similar), I think I'd try to automate some kind of online quiz they can go and take before they're even allowed to apply (which can include fizzbuzz or whatever little puzzles you want, checks for cheating etc) - then you can spend the phone screen or interview actually talking about interesting stuff which is in the problem domain they'll actually be tackling and finding out what they'd do with it. I just suspect these questions can be gamed and are not necessarily weeding out all the undesirables, just those who haven't done any homework on your hiring process.

I've seen people hired who talked a good game and could have passed these trivial tests without giving away that they'd just read up on likely ones, but were lacking in all the skills necessary to bring projects in on time and competently - that sort of mediocre candidate who lies about experience and knows the basics is far harder to stop.

It's interesting how different people react to IQ tests or little programming tests though - some people hate the implication of jumping through hoops, and that ability/knowledge can be reduced to such simple set of tests, and resent being subjected to them. Others really don't care and find them interesting or at worst trivial. I'd find it kind of weird trying to guess in an interview if someone who asks for say 'select the second highest number in this list' just wants a quick solution like list.sort[list.count-2] (which in the real world would be the first choice until or unless you are likely to hit major performance issues), or if they want you to write a trivial sorting algorithm for them and explain that you're familiar with the content of an undergraduate CS course.

It's also interesting seeing the range of different interview questions posted here as examples - some are trivial filters as you say, some are more involved, and some require knowledge of basic undergrad stuff, so I'm not completely sure that all of these puzzles are intended as completely trivial, there's always the temptation to elaborate them into some kind of exam (like those posted on HN a while back with maths questions) for esoteric knowledge which is frankly useless in most people's job.

For Mt Fuji, how you'd do it is less important than your first question(s): "Why?" and "What constraints?"

If it's to make space for a hyperspace bypass, I'll just let the Vogons handle it. If time is critical, how much mining equipment and how many many megatons do I have access to? What's the size of the evacuation zone? Is glowing gravel OK? There are some companies in Appalachia that have plenty of experience removing mountains, can I call on them?

FizzBuzz doesn't filter anyone, the people using it do. It is not difficult to explain the purpose of the exercise, and that easily resolves the problem for people looking for "clean" solutions.

I would think your computer science education would have prepared you to plan out such an algorithm. Sadly, its pretty had for a place to confirm you have subconscious skills by asking a question and then waiting for you to come back the next day with the answer once it popped into your head.