Live data from Hacker News

Why we don't hire programmers based on puzzles and tricks

37signals.com

291–300 of 460 posts

Re: Why we don't hire programmers based on puzzles and tricks

#291

Earlier quoted context omitted.

Anyone who's studied information theory would destroy that puzzle though. In the same way that someone with a physics degree would perform way above average on a "count the stars" question. Or someone whose dissertation was on computational biology would excel at a string-matching algorithm question. It is an example of the interviewer (or company/process) mistaking a specific problem as representative of all problem…

Anyone who's studied information theory would destroy that puzzle though. I understand this comment in a general sense--the scale's behavior represents two possible states, so you can use it to filter the answer space in O(log_2 N) ala binary search, and you can get away with O[(log_2 N) - 1] if you partition the answer space the right way. But what's the information theoretic interpretation of the problem?

It is a source coding problem. Kind of like a Huffman code. Depending on how theoretical or applied your information theory coursework was, you might have worked out simple codes like these in class, and that might be an advantage.

It's actually very similar to your tree reasoning above. The information theoretic outlook uses the probabilities of each of the N states to make a more efficient code, but since there are no probabilities in this case, the two approaches pretty much coincide.

Incidentally, there are three possibilities with each weighing -- scale tips right, scale tips left, scale stays balanced. So your logs should be base 3. In coding language, you are transmitting ternary valued symbols.

And, you should dispense with the O() notation, because the lead constant matters very much in this puzzle, and it happens to be 1.

Re: Why we don't hire programmers based on puzzles and tricks

#292
post #167
post #114

Earlier quoted context omitted.

I don't know if this is a Silicon Valley thing or what, but the idea that someone would basically do a day's unpaid work on projects they know nothing about with three different people is just about the most foreign thing I can think of. That being said, I've never done any pairing so maybe it would work better in that type of environment.

A job interview is going to take at least a day of your time anyway. What does it matter if that time is spent programming, talking over coffee, sketching on a whiteboard, or whatever?

Well with a "regular" interview, it's pretty unlikely that the company is going to make any money off of the direct output of the interview. When helping them develop a feature or product, it's slightly different.

Re: Why we don't hire programmers based on puzzles and tricks

#293
Just my 2 cents on this issue - writing code on a whiteboard isn't always about the code you write.

It's not about the actual code you produce in the interview, it's about talking through your thought process, your ability to reason while under pressure, and your attitude of whether or not "this is beneath me - wah!".

I've been in several interviews where I had to write code. I made some mistakes, some things I plain out didn't know (for my first job, I had no clue how to write any JavaScript, and I explained that it was just something I didn't know how to do, but that I'd be willing to learn it as required). I still got hired from all of them. So, if you think coding tests during interviews are about producing flawless code, they're not. They're to see how you think - and that is profoundly important as a developer.

I would also argue that a developer's existing skills are the least important factor when deciding whether or not to hire someone. Every developer works as part of a team and has to collaborate and communicate with others in that team. If you want to hire a developer, you need to see how well they communicate complex technical problems to non-technical people.

Re: Why we don't hire programmers based on puzzles and tricks

#294

Earlier quoted context omitted.

Anyone who's studied information theory would destroy that puzzle though. I understand this comment in a general sense--the scale's behavior represents two possible states, so you can use it to filter the answer space in O(log_2 N) ala binary search, and you can get away with O[(log_2 N) - 1] if you partition the answer space the right way. But what's the information theoretic interpretation of the problem?

It is a source coding problem. Kind of like a Huffman code. Depending on how theoretical or applied your information theory coursework was, you might have worked out simple codes like these in class, and that might be an advantage. It's actually very similar to your tree reasoning above. The information theoretic outlook uses the probabilities of each of the N states to make a more efficient code, but since there are…

Thanks! Haven't taken a formal information theory class yet, but I'd like to soon.

Re: Why we don't hire programmers based on puzzles and tricks

#295
post #263

Earlier quoted context omitted.

I think the point of the article is this: Trivial puzzlers like:write a fizzbuzz program, swap two variable contents without using a temp variable, how do you count the stars in the universe Just encourage people to game the system. It's impossible to get real insight from them, because you can't differentiate between those who worked it out, and those who have read about it. If you're in an interview by now and some…

Come to think of it, swapping two variables in place is a horrible question - because you can't really do it! A statement like a = a + b requires the (invisible) presence of a register where a + b (or xor or whatever) is placed right before being written back to a. But then you might as well use this register directly for the swap instead of dabbling around with one-to-one functions.

python: a, b = b, a

Re: Why we don't hire programmers based on puzzles and tricks

#296

Earlier quoted context omitted.

One thing you're testing for is how well their brain works with an excess of adrenaline floating around in their body. All of the best interviewers I've known take great pains to make the interviewer feel comfortable and chatty - after all, they aren't going to be filled with interview anxiety when they're actually working there. A minority of your applicants are going to perform badly in that situation because the a…

"A minority of your applicants are going to perform badly in that situation because the adrenal glands will effectively reduce the output prefrontal cortex below an acceptable level" I have an off topic question. I don't see what the problem being discussed has to do with how glands reduce activity in certain parts of the brain. Sounds to me like a fancy way of saying 'people don't perform well under stress'. You cou…

I think it's a little more specific than "people don't perform well under stress." A software development job may well entail stressful situations—for example, "The founder got interviewed on Oprah and the servers are melting!" But that kind of stress is different from a interview stress. There are definitely folks that will step up to the challenge of melting servers, but go to pieces under the social stress of an interview.

Now, maybe getting into the biological mechanism by which they go to pieces is too specific, and trotsky could have communicated more effectively by eliding it. But it's an important point, and I don't think it reduces to just "people don't perform well under stress."

Re: Why we don't hire programmers based on puzzles and tricks

#297

Earlier quoted context omitted.

All this teeth-gnashing regarding programming and logic puzzles are simply to cover up the simple fact that most people (even around here) are insecure about their programming skill, and more-so their intelligence. The fact is that these types of questions are indeed backed up by the very research you cite, namely that a work-sample test and general mental ability are the only reliable predictors of job performance.…

> most people (even around here) are insecure about their programming skill, and more-so their intelligence. > Instead of writing endless blog posts and internet comments about how useless these tests are ... just get better at them! You'll be better off for it. This doesn't hang together. Will getting "better" at programming tests & logic puzzles make you a better programmer? Make you more intelligent? If not, doesn…

It depends on your strategy for "getting better at these types of problems". You can cram and memorize all the algorithms you can, and that may allow you to pass some filters that you otherwise would not have. This strategy has its uses: besides being offered more jobs, you now have knowledge of these algorithms in your head and thus more tools in your toolbox.

Ideally, the candidate would learn about these "exotic" data structures, understand their properties, learn the cases where they are useful, and most importantly develop mental heuristics to recognize patterns of problems and appropriate classes of solutions. This will also help you pass these filters, and in the process will absolutely make you a much better programmer.

Re: Why we don't hire programmers based on puzzles and tricks

#298
post #145

Earlier quoted context omitted.

I think the point of the article is this: Trivial puzzlers like:write a fizzbuzz program, swap two variable contents without using a temp variable, how do you count the stars in the universe Just encourage people to game the system. It's impossible to get real insight from them, because you can't differentiate between those who worked it out, and those who have read about it. If you're in an interview by now and some…

It's impossible to get real insight from them, because you can't differentiate between those who worked it out, and those who have read about it. The fact that they're involved enough in the industry to know about fizzbuzz is still a strong signal. It will only come up if participate in some sort of developer community, which, I suspect, is something that the completely incompetent don't do, as a rule. Once again, it…

It will only come up if participate in some sort of developer community, which, I suspect, is something that the completely incompetent don't do, as a rule.

A great thing a small company can do to find people is sponsor a users group of some sort. If your company does Java start a JUG, Ruby start a RUG, etc... For the price of some pizza you can informally meet people that by the nature of showing up give a strong signal as people you may want to talk to when looking for your next employee.

Re: Why we don't hire programmers based on puzzles and tricks

#299
post #44

Sigh. This is the issue that will just never die. Let's just summarize the points: - Not everyone can produce real-world code. Most of what we do for a living belongs to our employer. Side projects, particularly in the US, can be an issue for IP reasons (California is somewhat of an exception here); - Side projects, even open source projects, can be of questionable "real world" value; - I've personally encountered ma…

I don't know if we've gotten there yet, but Fizz Buzz specifically may not always be an effective weeding-out tool: once enough people have heard of it, they may prepare specifically for it. Perhaps a better approach would be to come up with your own small test of comparable difficulty, sort of like a blog that has a custom but otherwise extremely hackable way of testing if you're a real person.

Re: Why we don't hire programmers based on puzzles and tricks

#300
post #237

Earlier quoted context omitted.

> a programming puzzle is the intersection of a work-sample test with one question from an IQ test In the worst case this is true. But ideally the question will have various components, and various levels of difficulty. I'm reminded of a blog posted here some time ago that had the candidate split a string based on a provided dictionary of words (or something to that effect). The best candidates produced maximally eff…

That still doesn't help the candidates who just don't see the trick. What if you know memoization and dynamic programming, but it just doesn't occur to you that day, that to correctly split on spaces (not adding empty tokens at the beginning and end of the string, etc.) you need to use a state machine? You could give the candidate that "step" of the solution and continue on to see if they know the rest, but the candi…

If your point is that these sorts of tests don't have any statistical power, I would whole-heartedly agree. Certainly you have to be careful with the types of conclusions one draws from the results of such tests. But they are not useless. The probability that a candidate will work out after having solved the problem with the best and most efficient solution is certainly much higher than if she only gave a middle-of-the-road solution. They are useful as positive filters.

And just to be a unnecessarily technical: if you accept the premise that the combination of appropriate CS knowledge and intelligence increases the likelihood that a candidate will successfully answer the question, then the fact that they answered the question successfully increases proportionally the probability that the candidate has sufficient CS knowledge and IQ (good ol' Bayes' theorem). This then increases the probability that the candidate will be successful on the job (citation in ancestor comment). That was a long winded way of justifying these types of questions as a positive filter.

Post reply on HN