Live data from Hacker News

Why we don't hire programmers based on puzzles and tricks

37signals.com

231–240 of 460 posts

Re: Why we don't hire programmers based on puzzles and tricks

#231
post #114
post #80

Earlier quoted context omitted.

I'm not sure if I'm that guy or not... but anyhow, here's the hiring process that's given me some good results. Note that this requires that your team does pairing. Interviewing answers three questions: Can they do the job? Do they get along with the team? And, can they adapt when the game changes? Before your phone interview, let them know that you want to do a short screen-sharing and pair-programming exercise via…

I don't know if this is a Silicon Valley thing or what, but the idea that someone would basically do a day's unpaid work on projects they know nothing about with three different people is just about the most foreign thing I can think of. That being said, I've never done any pairing so maybe it would work better in that type of environment.

I doubt that work important to the company is done this way. Otherwise, they'd expose proprietary information. Of course, I could be wrong.

Re: Why we don't hire programmers based on puzzles and tricks

#232
post #150

The best interview I ever had, I was given a problem to solve with a system, a whiteboard and 20 minutes. I sketched out a quick ERD, and explained the high level architecture of the system. No code was written -- it was assumed that I could implement said system if I could design it. The worst job interview grilled me on low level algorithms (I'm not a CS grad) and wanted me to sketch out a quick sort. Of course, I…

A programmer who is unable to program a Quicksort implementation is like a programmer who can't program anything relatively complex.

Re: Why we don't hire programmers based on puzzles and tricks

#233
post #44

Sigh. This is the issue that will just never die. Let's just summarize the points: - Not everyone can produce real-world code. Most of what we do for a living belongs to our employer. Side projects, particularly in the US, can be an issue for IP reasons (California is somewhat of an exception here); - Side projects, even open source projects, can be of questionable "real world" value; - I've personally encountered ma…

> Not everyone can produce real-world code. Most of what we do for a living belongs to our employer. Side projects, particularly in the US, can be an issue for IP reasons (California is somewhat of an exception here)

Says who? My side project that I do on my own time doesn't use the same language as my day job nor does it tackle the same problem domain. How can this be construed as an intellectual property issue?

Re: Why we don't hire programmers based on puzzles and tricks

#234
post #142
post #103

This is something to think about. The interview process is so badly gamed that these days its kind of a ceremony. In a recent interview, I gave a candidate a practical problem. He was allowed to use the sed man page, and then I gave him a file and search/replace problem. Nothing much! The candidate fails! Instead he asks me if there is going to be a algorithm/data structure interview. I told him, I don't know but as…

I do something like that. My first test is ask to reverse a string (not using the reverse function!) in any language and/or pseudocode. Is incredible how badly the test goes with the people! I interview for myself or others more than 100 people (of any background, including university, tech schols...) and I think only 10/12 people do it correctly - barely- To make it more fair, I always let them alone. And give inter…

This is great. My first thought was "perfect, Haskell!"

reverse "whatever"

"No reverse function? but we can use the internet? The Prelude source probably has the best implementation of a reverse."

foldl (flip (:)) [] "whatever"

Re: Why we don't hire programmers based on puzzles and tricks

#235
post #114

Earlier quoted context omitted.

I don't know if this is a Silicon Valley thing or what, but the idea that someone would basically do a day's unpaid work on projects they know nothing about with three different people is just about the most foreign thing I can think of. That being said, I've never done any pairing so maybe it would work better in that type of environment.

>I don't know if this is a Silicon Valley thing or what, but the idea that someone would basically do a day's unpaid work on projects they know nothing about with three different people is just about the most foreign thing I can think of. Yep. That basically amounts to an application fee.

If you think it costs you something to be interviewed (thus your suggestion about an application fee) it certainly costs the company at least that much if not more. They are paying their staff to work with you on something that likely will not produce direct value for the company (usually a throwaway project or trivial task)

Plus they will spend time before and after the interview discussing your qualifications, performance etc. It comes out about even.

Re: Why we don't hire programmers based on puzzles and tricks

#236
post #80

Earlier quoted context omitted.

I'm not sure if I'm that guy or not... but anyhow, here's the hiring process that's given me some good results. Note that this requires that your team does pairing. Interviewing answers three questions: Can they do the job? Do they get along with the team? And, can they adapt when the game changes? Before your phone interview, let them know that you want to do a short screen-sharing and pair-programming exercise via…

I'm curious if you wouldn't mind sharing a few more details of your experience pairing a candidate in the office with your senior engineers. - Are the candidates able to help make meaningful contributions to whatever the senior engineers are working on without having the background and domain knowledge? - Do the employees have to spend a significant amount of the 2-3 hours with the candidate bringing them up to speed…

I havent done pairing interviews but I have tried giving candidates some started code and having them develop a solution based on that.

One piece of logistical advice here: keep different kinds of keyboards/IDEs available. I use a mac and I interviewed a candidate who wasnt comfortable with the mac keyboard. In an interview situation where a person is already slightly nervous the last thing you want is for them to be worrying about keyboard shortcuts!

Re: Why we don't hire programmers based on puzzles and tricks

#237
post #187

Earlier quoted context omitted.

> A programming puzzle is simply the intersection of a work-sample test and an IQ test. No: a programming puzzle is the intersection of a work-sample test with one question from an IQ test. IQ tests get a good handle on your intelligence by testing you over a wide range of near-trivial questions, each known to similarly measure intelligence, and then adding the scores up. By giving you so many opportunities to succee…

> a programming puzzle is the intersection of a work-sample test with one question from an IQ test In the worst case this is true. But ideally the question will have various components, and various levels of difficulty. I'm reminded of a blog posted here some time ago that had the candidate split a string based on a provided dictionary of words (or something to that effect). The best candidates produced maximally eff…

That still doesn't help the candidates who just don't see the trick.

What if you know memoization and dynamic programming, but it just doesn't occur to you that day, that to correctly split on spaces (not adding empty tokens at the beginning and end of the string, etc.) you need to use a state machine?

You could give the candidate that "step" of the solution and continue on to see if they know the rest, but the candidate will likely be feeling a loss of confidence. They'll think "hey, if I couldn't answer the 'easier' part, why should I be able to answer the 'hard' part?" and then their mind will lock up even harder when they search it for the next step.

This is why IQ tests are organized as a series of orthogonal questions. No question is dependent on any other question, so you can just move on quickly from a failure and prove your worth on another question.

In fact, on a usual IQ test, you're even given five or six "presentations" of each question, each slightly different--like math homework questions with different numbers inserted. It happens quite often that you'll get stuck on, say, recognizing how one specific pattern of shapes go together, but will be able to see a different pattern instantly.

Interviews do the opposite of this; they are presented as if intelligence is a linear quantity, and if you fail on any part of a puzzle, you couldn't possibly be able to solve a "harder" part. In other words, interviewers are using the "error" theory of intelligence, rather than the "bugs" theory[1].

[1] http://news.ycombinator.com/item?id=5233497

Re: Why we don't hire programmers based on puzzles and tricks

#238
post #114

Earlier quoted context omitted.

I don't know if this is a Silicon Valley thing or what, but the idea that someone would basically do a day's unpaid work on projects they know nothing about with three different people is just about the most foreign thing I can think of. That being said, I've never done any pairing so maybe it would work better in that type of environment.

I doubt that work important to the company is done this way. Otherwise, they'd expose proprietary information. Of course, I could be wrong.

Spending a day doing useless work is worse.

Re: Why we don't hire programmers based on puzzles and tricks

#239
Comments are closed on the article and I wanted to thank David.

I'm one of those who freeze up like a deer in headlights when asked to do a quiz. But have a degree in CS, have made several enterprise level packages by myself, and have been programming for 31 years -- so am fairly confident that I can program.

Re: Why we don't hire programmers based on puzzles and tricks

#240

Earlier quoted context omitted.

I generally agree, most logic questions are useless, but Big-O notation I use every day. Programmers who don't understand the performance difference between Array#include? and Set#include? in ruby are dangerous!

I'm curious to know where you use the notation in daily programming. In code comments? Discussions with other developers? Other documentation? I use the spirt of Big-O almost daily, but do the actual calculations rarely, and basically never use the notation. With some experience, you quickly get an intuitive feel for execution time, which is good enough in most cases.

The last time I talked about Big-O notation was three days ago, when talking about whether or not inserting/deleting from a list should be done online or queued up:

  Zarel: and for insertions, aren't they O(nm) for n insertions into a list of size m either way?
  Cathy: inserting n elements into a list of size m when done with queueing requires at most m iterations
  Cathy: when not done using a queue, it requires at most n*m iterations
  Cathy: a factor of n is the difference
  Cathy: of course, n might be pretty small, depending on how many insertions happen in the debounce wnidow
  Cathy: n might be 10-15
  Zarel: how do you insert n elements into a list of size m in m iterations?
  Cathy: iterate over the list of m elements
  Cathy: well, i guess you'd need to sort the list of n elements first
  Zarel: oh, I see, I forgot about sorting the list
  Zarel: so it'd be O(n+m)
  Zarel: which is admittedly much faster for our purposes
  Zarel: well, considering n is likely to be approximately 5, "much" might be an exaggeration
  Cathy: in any case, i agree that inserting/deletion approach is the right way to do this
  Zarel: incidentally if we wanted we could apply certain optimizations to improve an insertion from O(n)
  Zarel: for instance, binary search
  Cathy: it's not immediately obvious whether that is faster than the sorted queued
  Zarel: actually, that's not a bad idea regardless
  Cathy: but it's certainly faster than the naive single insertion
  Zarel: I mean, O(n log m) is significantly faster than O(n+m)
  Zarel: when n is ~5 and m is ~1500
  Cathy: true, and it actually makes it clear that using a queue is unnecessary
I don't know of a better way to discuss optimizing an inner loop than to use big O notation.
Post reply on HN