Live data from Hacker News

How to conduct a good programming interview

lihaoyi.com

51–60 of 188 posts

Re: How to conduct a good programming interview

#51
post #36

Interviews suck for both parties, a world without interviews would be a better world. Why can't we just hire someone based on his experience and projects he previously worked on? You can have a first month as a "trial" where both parties can cancel the contract. Why interviews suck is because they are very, very biased and subjective and also not a good way to estimate the actual performance of a candidate. Say no to…

The effort required to get someone new spun up on your project isn't trivial (unless what you're doing is trivial...). There is a lot to understand, both "hard" knowledge about the tech stack, product, specific features you're working on, customer use cases etc as well as the "soft" stuff, like communication style, the specific rhythm of the team and more often than not, reading a bit of politics. Never mind that this is stressful for the applicant (and would require them to quit their previous job for a perfectly insecure alternative), it takes a substantial effort on part of the existing team to do this.

A commonly mentioned number for application success rates are on the order of 1:100. Even if this process is 10x better, a team will have to welcome a new "apprentice" every month (and fire most of them again) to get a single new hire a year.

This is not obviously an improvement for either employer or employee.

Re: How to conduct a good programming interview

#52
Here's another disadvantage of whiteboard coding: it's inaccessible to blind people, and impractical (even more than usual) for people with only a little vision (like me). Such candidates will need to do the coding on a computer, so one might as well make that the default, as the OP recommends, and eliminate one variable and a possible source of bias. Of course, once a blind or low-vision programmer stays at a given company long enough, they'll probably be on the other side of the interview at some point. So for consistency among a team of interviewers, I imagine it would be best if all of them have their candidates do coding problems on a computer, not just one of them.

I'm curious about why whiteboard coding is still a thing in the first place. Is it a matter of convenience or cost? If so, surely that equation has changed now that we have ubiquitous, cheap, portable computers. Or do some interviewers really think that it's good to deny a candidate access to a real computer while solving a coding problem during the interview?

Re: How to conduct a good programming interview

#53

So reading over some of the questions posted on technical programming interviews I must say that I would definitely fail such an interview without studying for it for a month first. Having recently discussed job interviews of ex-colleagues that went to EBay, Facebook, Google, etc.. I wouldn't even pass the whiteboard test. However I'm shipping scaleable API's used by web and native apps, using Continuous Integration…

> I completely understand the need for these technical interviews and weed out obvious bad programmers and get experts on board but sometimes from a business point of view it just doesn't make sense.

The thing is that the main goal of the technical interview is not to hire all good candidates. The main goal is to avoid hiring bad candidates. A large high paying company like Facebook has plenty of candidates to choose from, and the main priority is avoiding expensive bad hires. The cost of failing to hire a good engineer is small - they can just take the next guy. The cost of hiring a bad engineer is large.

With this in mind, the process makes a lot of business sense. Someone who is capable of learning what's needed to do well on a technical interview has demonstrated a fairly high level of competence, and is (relatively) unlikely to be a really bad hire.

This filter does limit your applicant pool a little, but if you're Facebook, the largest chunk of your applicant pool are recent comp. sci. graduates. If they can't solve an algorithms problem reasonably gracefully then that's a bit of a red flag. For non-recent-comp-sci-graduates, a competent candidate really should be able to prepare pretty well in a month or two of spare time.

Facebook might miss out on hires like you who would likely be quite good at the job but aren't willing to put in the effort to prep for the interview, but that's a trade off they'll make happily if they minimize bad hires.

Smaller companies with lower pay and access to fewer qualified candidates would probably be ill advised to copy the Facebooks and Googles of the world though. I'll grant that.

Re: How to conduct a good programming interview

#54

While there are a variety of ways you can conduct a programming interview, the goals typically are the same: Will the candidate be able to write working code if they join the team? Can the candidate discuss code and problems with the people they'll be working with? Can the candidate reason about arbitrary problems and constraints? Is the candidate someone we would enjoy working with? All of these can be solved direct…

This is an interesting candidate-first approach, but I have some concerns:

This requires that candidates are experienced at the language or framework you use. Are you not prepared to take someone with different experience? What about graduates? What if their ability with one framework does not transfer to another that you end up assigning them to?

Not all bugfixes are equal. If you haven't solved it yourself before, how do you know how hard it will be? If you give candidates questions that vary in difficulty, how do you compensate for that?

Is the ability to dive into a codebase and fix (presumably) trivial bugs really what you should be optimizing for? Is it actually better than abstract algorithmic questions for determining if the candidate [can] reason about arbitrary problems and constraints?

How would you overcome your unconscious biases when answering the question Is the candidate someone we would enjoy working with?

Re: How to conduct a good programming interview

#55

> 5 minutes of resume discussion > 45 minutes of coding As anyone else found that a good resume discussion is often much better than the coding component? I've concluded that implementation difficulties, decisions made (tradeoffs, technologies, etc.) and the collaborative environment around projects, are much better signals than the code part of my interviews. If you've dealt with a broad range of tech and can ask th…

Yeah. If they can't tell me about what they've built before, and aren't a less-than-one-year-away-from-school new grad, I'm going to be wondering how they deal with real problems regardless of whatever they can do algorithm-wise.

If someone can talk about what approach they took, what other approaches they considered, what made it hard, what made it easy, how often the requirements changed, etc, then I'd rather ask them questions about that then spend the whole time in code. Just do some small spot checks to make sure they aren't taking credit for the work of others.

Re: How to conduct a good programming interview

#56

So reading over some of the questions posted on technical programming interviews I must say that I would definitely fail such an interview without studying for it for a month first. Having recently discussed job interviews of ex-colleagues that went to EBay, Facebook, Google, etc.. I wouldn't even pass the whiteboard test. However I'm shipping scaleable API's used by web and native apps, using Continuous Integration…

> I completely understand the need for these technical interviews and weed out obvious bad programmers and get experts on board but sometimes from a business point of view it just doesn't make sense. The thing is that the main goal of the technical interview is not to hire all good candidates. The main goal is to avoid hiring bad candidates. A large high paying company like Facebook has plenty of candidates to choose…

This is a great level headed summary of the whole CS degree vs craftsman argument, thank you.

Re: How to conduct a good programming interview

#57

Here is how to do a good coding interview if you are a recruiter: 1. Do your homework: know what projects drive the interviewee (what kind of things she likes to code, what recent projects has she been involved in), make sure they are a match for your company projects and goals 2. Ask the interviewee for a git repository and take a careful holistic look at it (commit messages and code, timespan, branches, merges etc.…

I think you missed the part where the OP said not all candidates are willing or able to work on open source projects.

Re: How to conduct a good programming interview

#59
post #54

While there are a variety of ways you can conduct a programming interview, the goals typically are the same: Will the candidate be able to write working code if they join the team? Can the candidate discuss code and problems with the people they'll be working with? Can the candidate reason about arbitrary problems and constraints? Is the candidate someone we would enjoy working with? All of these can be solved direct…

This is an interesting candidate-first approach, but I have some concerns: This requires that candidates are experienced at the language or framework you use. Are you not prepared to take someone with different experience? What about graduates? What if their ability with one framework does not transfer to another that you end up assigning them to? Not all bugfixes are equal. If you haven't solved it yourself before,…

After I proposed the idea, they said they liked it and were willing to give it a try. They offered 1.) slight changes to the UX, 2.) add a payments feature, or 3.) some back-end bugs that need to be resolved. They asked which of them sounds most appealing as a test.

This was perfect from my perspective, and after I signed the NDA I told them all of those sounded fine. I mentioned that fixing a bug is one of the best ways to get familiar with a new codebase, so I'd probably focus on those, but that I'd like to hear about the rest too. I can probably throw in some of the feature requests or UI tweaks.

That's a bit more candidate-driven than might be expected, but it's really wonderful so far. I feel like I have autonomy, while at the same time they'll get the data about whether I'm effective in the field.

The bias question is important, and I'm not quite sure the best way for us to collectively deal with it. All I know is that there seems to be far more bias in a traditional interview pipeline. One of my recent rejections came after being asked "A hammer and a nail cost $1.10, and the hammer costs a dollar more than the nail. How much does the nail cost?"

Obviously the point of that is to see how you think through a problem. But such problems are useless. They're abstracted from what we as engineers do in the field, and the candidate knows that their entire fate rests on how well they answer that question. Yet those are the kind of metrics real companies are currently using.

Re: How to conduct a good programming interview

#60

While there are a variety of ways you can conduct a programming interview, the goals typically are the same: Will the candidate be able to write working code if they join the team? Can the candidate discuss code and problems with the people they'll be working with? Can the candidate reason about arbitrary problems and constraints? Is the candidate someone we would enjoy working with? All of these can be solved direct…

Almost. Instead of doing this, take an old set of bugfixes from the tree, 1-3 of them, give the same bugfixes to every candidate, and come up with a standard rubric for judging the fixes. (If it's an open source project, synthesize 1-3 bugs, using the history of bugs on the project as a guidepost).

This is a "work sample test", and it's the gold standard for judging technical qualification. Firms should rely on them to the exclusion of subjective interviews.

Within reason, to a first approximation, every candidate gets the same problems (you may iterate over time). You can judge candidates apples-apples.

You don't need to pay candidates for their time, because they're not doing work for you; you already fixed these bugs.

Post reply on HN