Live data from Hacker News

How to conduct a good programming interview

lihaoyi.com

61–70 of 188 posts

Re: How to conduct a good programming interview

#61
post #35

Earlier quoted context omitted.

Could just be a terminology mismatch. Checking for hash existence should always be constant, but checking for key existence is generally only amortized constant (that is to say, not constant across all lookups) - he may have wanted you to talk about addressing schemes and degradation at high load factors?

There was no subtle argument. It only came up because I was on the whiteboard solving some simple word game that involved matching user input against a list of valid words (e.g. the Scrabble dictionary). When asked to explain the running time of the various operations, I said that we can store the dictionary in, well, a dictionary , and checking whether the user input is a valid word will be a constant time operation…

So likely thinking of it in terms of checking a tree based on characters instead of hashes of full words. Which is more viable would likely depend on what kind of memory constraints and dictionary size was being considered. Also very implementation dependent, what's the size of the hashes generated?

Edit: curiosity got me looking at this for Python, with an answer of "it depends, and both have correct aspects" based on Python's implementation [1]. The hashing varies by string length, the lookup is fixed unless there's a collision and probing, probing will be minimized by growing the dictionary as items are added. In any case, better than O(n) but worse than O(1) and with some possible slowdowns during loading unless you can set an expected final size during initialization.

Edit again: since I brought it up, turns out initialization doesn't really matter, resizing is not a speed factor. Relevant discussion starts from https://stackoverflow.com/questions/1298636/how-to-set-initi...

[1] http://www.laurentluce.com/posts/python-dictionary-implement...

Re: How to conduct a good programming interview

#62

So reading over some of the questions posted on technical programming interviews I must say that I would definitely fail such an interview without studying for it for a month first. Having recently discussed job interviews of ex-colleagues that went to EBay, Facebook, Google, etc.. I wouldn't even pass the whiteboard test. However I'm shipping scaleable API's used by web and native apps, using Continuous Integration…

> I completely understand the need for these technical interviews and weed out obvious bad programmers and get experts on board but sometimes from a business point of view it just doesn't make sense. The thing is that the main goal of the technical interview is not to hire all good candidates. The main goal is to avoid hiring bad candidates. A large high paying company like Facebook has plenty of candidates to choose…

>> The cost of failing to hire a good engineer is small

I'm not so sure about that.

https://en.wikipedia.org/wiki/Jan_Koum

"In September 2007 Koum and Acton left Yahoo and took a year off, traveling around South America and playing ultimate frisbee. Both applied, and failed, to work at Facebook."

"On February 9, 2014 Zuckerberg asked Koum to have dinner at his home, and formally proposed Koum a deal to join the Facebook board - 10 days later Facebook announced it was acquiring WhatsApp for US$19 Billion USD."

Re: How to conduct a good programming interview

#63
post #9

Earlier quoted context omitted.

I used to think that, but honestly after recently going through a job search I think it's untenable for most scenarios. As a candidate, I want to be able to interview with multiple companies simultaneously—if I had to do a trial week with each company, I'd never be able to get multiple parallel offers. There's also significant onboarding complexity which I hadn't factored in. One company had me do a trial period and…

True enough, but it sounds like the above model isn't for you. It's for everyone who is talented but doesn't have the option to go interview with multiple companies simultaneously. Most devs are in that position. I'd much rather commit to a single company that I want to work at and do this process to put me ahead of the other candidates. But that's not a decision that everyone feels the same about.

I get your sentiment. But this is quite problematic if you already have a job. It's not so easy to take a week or month off on short notice. What happens if you don't get your dream job? You can't just quit your current job.

Re: How to conduct a good programming interview

#64
post #60

While there are a variety of ways you can conduct a programming interview, the goals typically are the same: Will the candidate be able to write working code if they join the team? Can the candidate discuss code and problems with the people they'll be working with? Can the candidate reason about arbitrary problems and constraints? Is the candidate someone we would enjoy working with? All of these can be solved direct…

Almost. Instead of doing this, take an old set of bugfixes from the tree, 1-3 of them, give the same bugfixes to every candidate, and come up with a standard rubric for judging the fixes. (If it's an open source project, synthesize 1-3 bugs, using the history of bugs on the project as a guidepost). This is a "work sample test", and it's the gold standard for judging technical qualification. Firms should rely on them…

Yeah, I'm a big fan of your approach. Unfortunately it seems quite difficult to get anyone to do it. Most companies have interview pipelines that are well-established and immutable, so only new companies are inclined to try it. And even Triplebyte unfortunately fell back on a combination of abstract coding tests and amorphous "cultural fit" criteria.

It was a neat hack as a candidate to propose to the interviewing company that they do essentially the same thing as a work-sample test and hear them say yes. That only works if you propose they let you try your hand at doing their actual work. I wonder if it can be generalized?

Re: How to conduct a good programming interview

#65

So reading over some of the questions posted on technical programming interviews I must say that I would definitely fail such an interview without studying for it for a month first. Having recently discussed job interviews of ex-colleagues that went to EBay, Facebook, Google, etc.. I wouldn't even pass the whiteboard test. However I'm shipping scaleable API's used by web and native apps, using Continuous Integration…

> I completely understand the need for these technical interviews and weed out obvious bad programmers and get experts on board but sometimes from a business point of view it just doesn't make sense. The thing is that the main goal of the technical interview is not to hire all good candidates. The main goal is to avoid hiring bad candidates. A large high paying company like Facebook has plenty of candidates to choose…

But those big companies constantly complain about there not being enought people. And they also take people who passed hard tests and put them on positions that are way more rote or require different set of skills. And the geniuses then stay because of perks and social program, but talk about actual work as boring.

While there are many low skilled people who look for job, there are also many many companies/teams who can't admit to themselves that owerwhelming majority of positions they have is not that difficult and responsible less then genius could do them.

Re: How to conduct a good programming interview

#66
post #35

Earlier quoted context omitted.

There was no subtle argument. It only came up because I was on the whiteboard solving some simple word game that involved matching user input against a list of valid words (e.g. the Scrabble dictionary). When asked to explain the running time of the various operations, I said that we can store the dictionary in, well, a dictionary , and checking whether the user input is a valid word will be a constant time operation…

So likely thinking of it in terms of checking a tree based on characters instead of hashes of full words. Which is more viable would likely depend on what kind of memory constraints and dictionary size was being considered. Also very implementation dependent, what's the size of the hashes generated? Edit: curiosity got me looking at this for Python, with an answer of "it depends, and both have correct aspects" based…

It's also possible the interviewer meant linear time as a function of the length of the word, which would be true. Regardless, not being able to express this clearly is a sign of a poor interviewer which unfortunately the candidate takes the hit for.

Re: How to conduct a good programming interview

#67
It's good to think carefully about how you interview candidates. Most of the ideas in this post are good. You should definitely let candidates type answers on computers. You should definitely work to put them at ease, in an environment that approximates their actual working environment.

But really, this is window dressing. The problem is that technical interviews just don't work reliably. We have better ways to qualify candidates.

Rather than trying to read subjective, noisy signals from in-person interviews, give candidates a small battery of work-sample tests. These can be bug-fixes, small features, diagnostic challenges, or even specification writing assignments.

A properly designed work-sample challenge has a rubric: a set of instructions for scoring a result. Ideally, the rubric is straightforward enough for someone who hasn't met the candidate to quickly score their result. "Runs correctly". "Passes a battery of unit tests". "Includes appropriate documentation". Whatever is important for your team.

Work to give every candidate the same set of challenges, so you can track results over time. You'll find that after refining and iterating for a few months, you'll rapidly build confidence in the tests, to the point where in-person interviews become superfluous for technical qualification.

Working on technical qualifiers is stressful. Most candidates will perform more poorly on them than they would on normal work. You should build that into your rubric (though, really, data from candidates will ultimately determine how strict the rubric becomes).

Understanding how stressful qualifying work is, you should consider not requiring candidates to work on problems on-site. Instead, administer them online. Suggest the amount of time you expect the problems to take to finish, and offset that amount of time from the amount you demand on-site.

When you must interview, you should work hard to standardize the interview. Interviewers hate this. Everyone has their favorite questions. People want interviews to be conversational and adaptive. Interviewing isn't supposed to be fun for the interviewer; it's supposed to be reliable.

Re: How to conduct a good programming interview

#68

Earlier quoted context omitted.

> I completely understand the need for these technical interviews and weed out obvious bad programmers and get experts on board but sometimes from a business point of view it just doesn't make sense. The thing is that the main goal of the technical interview is not to hire all good candidates. The main goal is to avoid hiring bad candidates. A large high paying company like Facebook has plenty of candidates to choose…

>> The cost of failing to hire a good engineer is small I'm not so sure about that. https://en.wikipedia.org/wiki/Jan_Koum "In September 2007 Koum and Acton left Yahoo and took a year off, traveling around South America and playing ultimate frisbee. Both applied, and failed, to work at Facebook." "On February 9, 2014 Zuckerberg asked Koum to have dinner at his home, and formally proposed Koum a deal to join the Faceb…

It seems unlikely to me that as a full time Facebook engineer Koum would have still had time to create a multi-billion dollar startup.

It also seems unlikely to me that he would have been motivated to put in that work for an employer, or that Facebook would have given him the time or freedom necessary to do it.

If he had been hired, he'd have probably been a fine engineer, but there's really no reason to think he'd have added more value than the next guy. The skillset needed to create a successful startup is not the same as the skillset needed to be an effective engineer at a large company.

Re: How to conduct a good programming interview

#69
post #60

Earlier quoted context omitted.

Almost. Instead of doing this, take an old set of bugfixes from the tree, 1-3 of them, give the same bugfixes to every candidate, and come up with a standard rubric for judging the fixes. (If it's an open source project, synthesize 1-3 bugs, using the history of bugs on the project as a guidepost). This is a "work sample test", and it's the gold standard for judging technical qualification. Firms should rely on them…

Yeah, I'm a big fan of your approach. Unfortunately it seems quite difficult to get anyone to do it. Most companies have interview pipelines that are well-established and immutable, so only new companies are inclined to try it. And even Triplebyte unfortunately fell back on a combination of abstract coding tests and amorphous "cultural fit" criteria. It was a neat hack as a candidate to propose to the interviewing co…

I think asking candidates to do "real" work is bad for a bunch of reasons:

* It's intrinsically exploitative, even when compensated, because real work done under similar terms is paid far more highly than prorated salary (it also does weird things to salary negotiation, by extracting an up-front concession on rate).

* It requires different candidates to work on different problems, which means you're not getting a reliable record of how candidates do on the interview so you can tune it over time.

* Because the best candidates will generally refuse to work under these terms, most firms that do paid up-front work have a fast-path for elite candidates, which is an obvious process failure.

Re: How to conduct a good programming interview

#70
post #27

While there are a variety of ways you can conduct a programming interview, the goals typically are the same: Will the candidate be able to write working code if they join the team? Can the candidate discuss code and problems with the people they'll be working with? Can the candidate reason about arbitrary problems and constraints? Is the candidate someone we would enjoy working with? All of these can be solved direct…

They're compensated for their time, of course. Taking on paid work for another company, possibly a competitor, would put nearly anyone who did so in breach of their contract with their current employer, as well as complicating their tax situation. Not to mention the vacation time they would be burning on it. You will only be able to hire freelancers or people currently unemployed by this method. Do candidates set the…

I think it would be prudent for the interviewer to just say that "of course we will compensate for your time because this takes a couple of hours". If you happen to have a contract that forbids you from doing that, just say "Can't take any comp, I'm happy to do it for free". I don't think the key is compensation, but at least not assuming the subject will work for free. If you are the interviewer in that situation, give them the t-shirt and mug.
Post reply on HN