The Terrible Technical Interview • Hacker News Board

Re: The Terrible Technical Interview

#151

Post by fsk » Sun, Mar 22, 2015, 4:34 AM UTC

hibikir wrote: post #146

frostmatthew wrote: post #71

Earlier quoted context omitted.

> Try it out with some numbers. 10100 candidates, 100 are "good". What you're attempting to do works well for hypothetical drug testing[1] or terrorists but not for hiring developers (or anyone else). With the numbers you used you're proposing that less than 1% of all candidates are "good" - nobody would reasonably set the "good" threshold to include only the top 1% of developers. [1] https://en.wikipedia.org/wiki/Ba…

You are assuming that the percentage of people interviewing for a job are a good representation of the general programming population. That's not true at all. First, unless you really think we are terrible at hiring as an industry. So even if on a given day all developers that start looking for a job have a skill level that matched the average population, the good developers will find jobs faster, leaving the 4th, 5t…

If your hiring based on referrals and references and networking, you're hiring for a different skill set than programming ability.

Based on my experience, most employers don't hire better than "Pick a candidate at random".

Also, if you had an employee that was super-brilliant, why would you tell someone else so they can hire them away from you?

Re: The Terrible Technical Interview

#152

Post by sytelus » Sun, Mar 22, 2015, 4:52 AM UTC

fsk wrote: post #5

False negatives are a lot worse than most interviewers think. (False negative = you pass on a good candidate, false positive = you hire someone who turned out to be unqualified) If a "good" candidate is a 1-in-100 find, then each false negative means you have to look at another 100 candidates. Also, if you decrease your false negative rate by more than you decrease your false positive rate, you're actually hiring MOR…

No. Many companies have hiring rate of 10% [1] which means hiring 1 candidate may cost you 10 person-days assuming full day interviews for each candidate [2]. You can probably double that to cover the cost of phone screens, so let's say 20-person day of work per good hire. Assume you had 4 false negatives just because you are so aggressive and so a good hire ended up costing you 100 person days instead. This cost is still tiny compared to damages that a true negative would cause otherwise, i.e.,

(1) loss of entire person year or even two because annual reviews need to accumulate evidence for HR to fire (may be less time if you were startup without real "HR")

(2) amount of cleanup other people have to do after that new bad hire

(3) loss in moral for good people in your team who now perceives your hiring process at the company as "broken"

(4) delays + bugs introduced in product because actual work probably didn't got done or badly done despite of you filling up your headcount

(5) amount of money lost in salaries, signing bonuses, office space and benefits (typically > $200K)

(6) amount of productivity lost because of wasted time by good people in the team trying to "ramp up" your true negative

(7) emotional stress you caused to good people wondering them about their job stability and to managers who wasted their time in months of paper work and lot of explaining

(8) emotional stress you caused to your true negative being fired who had moved across the country for you, bought a house on mortgage and had 3 school going children

(9) Most likely, if you are big company, true negative didn't actually got fired because hiring manager never wanted to admit it. S/he was encouraged to join another team or role or even learned political tricks to get promoted contributing to ongoing bozo explosion[3]

(10) I could go on and easily justify probably 3X-10X loss compared the case if true negative was avoided

Footnotes:

1 - Many companies specify their hiring rate over total resume they received which is wrong. I'll use 10% as total number of full interviews that needed to be conducted which is average of 5%-15% at most companies.

2 - This is bad math. Assuming random trials, it would be actually 5 person-day on average but intuitive approach doesn't produce entirely bad results here so we will go with that.

3 - http://guykawasaki.com/how_to_prevent_/

Re: The Terrible Technical Interview

#153

Post by tedunangst » Sun, Mar 22, 2015, 5:15 AM UTC

tptacek wrote: post #94

StevePerkins wrote: post #87

Earlier quoted context omitted.

I'm not sure that you fully read the parent comment. I could care less about their Subversion expertise. My company doesn't even use Subversion anywhere. However, if you list " 10 years experience with " on your resume, then you should absolutely be prepared to discuss your experience with at a high level. Not anal minutia or contrived trick questions, but certainly you should be able to respond to an open-ended ques…

There is no less productive form of interview than the "resume validation interview". Resumes are practically useless in the best case. Here, you seem to propose paying a premium for candidates who can properly estimate their facility with VCS systems and then cogently discuss that estimation in an interview . That can't possibly be a skill relevant to building software on your team!

"Can you discuss what you've done?" seems like a skill likely to be highly relevant to team software building.

Re: The Terrible Technical Interview

#154

Post by fsk » Sun, Mar 22, 2015, 5:17 AM UTC

sytelus wrote: post #152

fsk wrote: post #5

False negatives are a lot worse than most interviewers think. (False negative = you pass on a good candidate, false positive = you hire someone who turned out to be unqualified) If a "good" candidate is a 1-in-100 find, then each false negative means you have to look at another 100 candidates. Also, if you decrease your false negative rate by more than you decrease your false positive rate, you're actually hiring MOR…

No. Many companies have hiring rate of 10% [1] which means hiring 1 candidate may cost you 10 person-days assuming full day interviews for each candidate [2]. You can probably double that to cover the cost of phone screens, so let's say 20-person day of work per good hire. Assume you had 4 false negatives just because you are so aggressive and so a good hire ended up costing you 100 person days instead. This cost is…

Hiring rate of 10%? You mean you hire 10% of all people who submit a resume? Or 10% of all people who come in for an on-site interview? Those are two different things.

Why should it take 2 years to fire someone? That sounds like a corporate bureaucracy problem.

Re: The Terrible Technical Interview

#155

Post by jqm » Sun, Mar 22, 2015, 5:20 AM UTC

foz wrote: post #17

In my organization, we start by looking at what the candidate has actually done. We ask for a github profile, and any side projects they might have worked on. If the CV and body of work look interesting, we go on to a phone screen - general questions, clarifying points on the CV, explaining the job, answering questions. For the on-site interview, we ask candidates to bring their laptop. We advise them to use a typica…

I like this approach. It's pragmatic. After all, it's not what a person knows (nor who they know) that produces results. It's what they do.

Re: The Terrible Technical Interview

#156

Post by jqm » Sun, Mar 22, 2015, 5:26 AM UTC

jimbobimbo wrote: post #108

I've been on both sides of the process and in both cases I prefer the whiteboard. As an interviewer, I simply don't have an hour or two to evaluate someone's project. I want to see a strong resume and ask a few questions to understand whether the resume is real or not. A simple whiteboard question or two is great for that. You can always see whether the person is capable of writing code and it always brings up questi…

If you don't have an hour or two as an interviewer, you probably aren't getting very good results. This is a pretty small investment of time considering what is at stake.

Re: The Terrible Technical Interview

#157

Post by tedunangst » Sun, Mar 22, 2015, 5:29 AM UTC

frostmatthew wrote: post #116

fsk wrote: post #85

Earlier quoted context omitted.

Even if you set the "good" percentage to 10%, too high a false positive rate will still ruin your results. Based on the people I've worked with over the years, I say that the actual skill distribution is: 5% toxic - These are the people who will ruin your business while deflecting blame to other people. 25% subtractors - These are the people who need more attention and help than the amount of work they get done. In t…

> 9% above average There is no reasonable definition of average that would only allow for 9% above that (or 10% including the 1% you marked as brilliant). Average is usually considered as either the 50th percentile (in which case you would have ~50% above this) or some middle range (e.g. 25th - 75th percentile). Since you said 60% are average we'll consider an appropriate range as average, the 20th - 80th percentile.…

Consider the set ( -10, 4, 4, 4, 4, 4, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 20, 50).

It has 20 elements and an average of 10. 5% are toxic. 25% are below average. 60% are average. 10% are above average.

Re: The Terrible Technical Interview

#158

Post by sytelus » Sun, Mar 22, 2015, 5:46 AM UTC

fsk wrote: post #154

sytelus wrote: post #152

Earlier quoted context omitted.

No. Many companies have hiring rate of 10% [1] which means hiring 1 candidate may cost you 10 person-days assuming full day interviews for each candidate [2]. You can probably double that to cover the cost of phone screens, so let's say 20-person day of work per good hire. Assume you had 4 false negatives just because you are so aggressive and so a good hire ended up costing you 100 person days instead. This cost is…

Hiring rate of 10%? You mean you hire 10% of all people who submit a resume? Or 10% of all people who come in for an on-site interview? Those are two different things. Why should it take 2 years to fire someone? That sounds like a corporate bureaucracy problem.

10% of total number of full interviews that needed to be conducted. At most publicly listed companies, firing can't be done without accumulation of enough evidence (what HR usually refers to as "paper trail"). This is true even when employment was "at will" mainly because of legal liability (for example, fired employee can claim that he was a victim of XYZ) and bad PR it can generate. I think Facebook is (or was) probably rare exception in aggressive firing and not sure how they managed it. Most companies also require annual review to be in place before firing occurs. Typically hiring manager would avoid firing within first year because that usually looks very bad on them. Most true negatives don't get fired until hiring manager changes or years pass by. At startups things are obviously different. Resources are scarce and true negative probably won't survive beyond 6 month or in worse case beyond a year. But still that's a significant period to cause enough of hemorrhaging for a true negative.

Re: The Terrible Technical Interview

#159

Post by jqm » Sun, Mar 22, 2015, 5:48 AM UTC

yason wrote: post #55

The more I interview the less I tend to give weight to the technical part and the more I focus on evaluating how well I get along with the person, how easily and naturally we can discuss programming topics, and do we happen to converge on building a rapport. People can learn technical stuff and people can grow professionally but the chemistry is harder to fix. And, unlike some programmers might see it, even code is c…

Well... that's certainly one approach. Hire people because you like them. Though it probably isn't the best approach for getting results.

Re: The Terrible Technical Interview

#160

Post by danans » Sun, Mar 22, 2015, 5:56 AM UTC

Articles demonizing coding interviews always attack strawmen that are the worst kind of interview questions: 1) Trivia questions (i.e. "what is the name of the method to do X in Java") 2) Questions with precisely 1 solution (you either get the one solution or you don't).

These are indeed useless technical interview questions.

But that doesn't meant there aren't good technical coding interview questions. I tend to favor the kind that present a candidate with some example data and a set of constraints or patterns, and ask them to write code that analyses the data for those patterns or constraints and reports them. This sort of problem is ubiquitous in my domain. Importantly, I always choose problems for which there are multiple possible attack angles, not just one. And I don't give a hoot about syntax errors, or what language they use.

This sort of question gives you a good sense about the candidate's analytical capability (breaking down the "word problem"), and their ability to translate their problem-solving thought process into code. Because there are always multiple angles of attack, candidates have some leeway to exercise creativity.

In the end, it's not terribly important to me that they get the optimal solution. I do care whether they demonstrate strong analytical capability in the literal sense, meaning they can decompose the problem and the associated programming exercise into their logical parts and implement them. I also look for good communication skills in the questions they ask when reasoning through the problem - this is something that only an in-person technical interview can reveal, AFAICT.

There are probably many smart candidates that don't do well on these questions because they're just having an off day, or nervous, or don't perform well under pressure. I sympathize with them, because I've been there and felt all of the above.

But if the goal of a technical interview is to assess a candidate's analytical and coding abilities, and their ability to do both simultaneously, there is no shortcut I know of to just giving them a role-relevant problem to work on.

EDIT:wording.