Live data from Hacker News

We analyzed thousands of interviews on everything from language to code style

blog.interviewing.io

91–100 of 181 posts

Re: We analyzed thousands of interviews on everything from language to code style

#91
post #9

Graphs have been fixed! Sorry about that, HNers.

There's something awry about the "fewer code execution errors" graph: I count an odd number of columns when they ought to be in pairs.

Clicking on the graph to go to plot.ly and viewing its "data" tab, it looks like there's a blank X value for:

         text                                                                           y                   x
    0    bucketed_success_rate: -0.02
pct: 0.1
`Interviewer Would Hire`: False 0.0987654320987654

Re: We analyzed thousands of interviews on everything from language to code style

#92
A lot of this article rings true with my experience. And I agree with the comments dinging live coding tests - those are the worst. I can't code effectively unless I'm calm and can concentrate, and these things are almost designed to get you off-balance.

Even more galling when you have a healthy GitHub portfolio that they refuse to even look at in favor of a quiz (this has happened recently).

Re: We analyzed thousands of interviews on everything from language to code style

#93
post #5

> Furthermore, no matter what, poor technical ability seems highly correlated with poor communication ability – regardless of language, it’s relatively rare for candidates to perform well technically but not effectively communicate what they’re doing (or vice versa), largely (and fortunately) debunking the myth of the incoherent, fast-talking, awkward engineer. My interpretation of this is that interviewees who can c…

I do this frequently and I utilize their resume to power the code.

Someone says "I am a 9/10 in JS" I give them something that uses the animation frame api, the dom api, and a few more javascript APIs that I personally am not fully comfortable saying that I am a 9/10 on (been working with JS for over ten years) and see how they decipher what it is doing, what jumps out, and how they would review it / give feedback.

9/10 times they don't actually understand Javascript.

Re: We analyzed thousands of interviews on everything from language to code style

#94
post #24

Earlier quoted context omitted.

That's actually how I conduct interviews. It's been immensely successful at weeding out candidates within 5 minutes; although most interviews are 45 - 60 minutes. Basically, give them less than ten lines of code, ask them what it does, where are a couple bugs, ask what would you name the function, etc. Then we talk about how to improve it. I'd say, less than 20% of people I interview pass. It's actually amazingly low…

"Half the people don't even tell me what they are thinking." Maybe they are introverted or simply need to think before they talk. Around 50% of people are introverted (less in usa). Many people are like that and simultaneously quite skilled. Moreover, some environments punish errors, so people who worked/studied there tend to be conditioned to think before talking. "And no matter how many times I try to work with the…

Generally to get candidates who are like this to open up, I simply assert:

"Tell me what you are thinking about right now"

"What is jumping out at you for you to be stalling?"

"Did you see something?" --> "Why?" --> "You seem to be quiet and deep in thought, what are you thinking about?"

etc...

Re: We analyzed thousands of interviews on everything from language to code style

#95

A lot of this article rings true with my experience. And I agree with the comments dinging live coding tests - those are the worst. I can't code effectively unless I'm calm and can concentrate, and these things are almost designed to get you off-balance. Even more galling when you have a healthy GitHub portfolio that they refuse to even look at in favor of a quiz (this has happened recently).

To note a few things regarding the interview and hiring process (I do a lot of it, have done about 20 in the past two days), the reason for the quiz and not using your portfolio (unless its extremely outstanding and you are fine doing a live coding questionnaire) is so that we can judge all the candidates equally and fairly.

It gives a common baseline to judge. Each candidate does the same thing and we have a good idea of what we are looking for, the rest of it tells us how you think, how you approach your work, organize your work, and best of all? You can compare that to how others do it.

Now, not to say that's everyone, that is what we use it for.

---

Before, as an Engineer / Manager, I hated doing "live coding" tests when it wasn't relevant. For example doing "algorithms" or "palindrome" or "sliding window dns" or "O(n)" examples when you're doing front-end or a management position screams to me that the people doing the interviewing don't know what they actually want.

Instead quizzes or live coding that are relevant like "tell me how to access all the elements in this particular element and traverse the children to apply some styling" is much more relevant and will show me the thought process, their ability to retain information, and their recall. It also shows communication ability when they get stuck and ask for help or use me as a sounding board.

It's not always about your implementation, but how you handle the situation and communicate

Re: We analyzed thousands of interviews on everything from language to code style

#96
post #24

Earlier quoted context omitted.

"Half the people don't even tell me what they are thinking." Maybe they are introverted or simply need to think before they talk. Around 50% of people are introverted (less in usa). Many people are like that and simultaneously quite skilled. Moreover, some environments punish errors, so people who worked/studied there tend to be conditioned to think before talking. "And no matter how many times I try to work with the…

Generally to get candidates who are like this to open up, I simply assert: "Tell me what you are thinking about right now" "What is jumping out at you for you to be stalling?" "Did you see something?" --> "Why?" --> "You seem to be quiet and deep in thought, what are you thinking about?" etc...

Yeah, that's great if you're NT and all that, but if you're interviewing someone who is anxious about the situation this might only add to the pressure, creating a melt down situation making it impossible for them to think.

In a normal work environment where he or she is left to think freely the the same candidate might excel.

Re: We analyzed thousands of interviews on everything from language to code style

#97
post #85
post #82

Earlier quoted context omitted.

If someone can't express himself, most likely that person won't work well in the team.

Some people take a bit of time and familiarity to warm up.

True. I am not sure if I want to take the risk though. I don't want to be a friend or tell jokes but i want to be able to talk technical stuff.

Re: We analyzed thousands of interviews on everything from language to code style

#98
post #77

Earlier quoted context omitted.

Isn't that why a lot of companies do tech phone screens?

If it's because of anti-discrimination laws, can a technical phone screen be short? You could discriminate based on voice too so I'd think that'd still be an issue.

I think the advantage is that the phone screening is much shorter than an hour, for everyone.

And "you haven't passed this interview" is less suspect than "I ended this interview early"

Re: We analyzed thousands of interviews on everything from language to code style

#99
post #24

Earlier quoted context omitted.

"Half the people don't even tell me what they are thinking." Maybe they are introverted or simply need to think before they talk. Around 50% of people are introverted (less in usa). Many people are like that and simultaneously quite skilled. Moreover, some environments punish errors, so people who worked/studied there tend to be conditioned to think before talking. "And no matter how many times I try to work with the…

Generally to get candidates who are like this to open up, I simply assert: "Tell me what you are thinking about right now" "What is jumping out at you for you to be stalling?" "Did you see something?" --> "Why?" --> "You seem to be quiet and deep in thought, what are you thinking about?" etc...

That sounds awful, you are actively sabotaging someone by ruining their concentration while they're trying to solve a bug in unfamiliar code under time pressure. This is like having a manager requesting status updates every 10 minutes when you're trying to fix a production failure.

Even if I solve the problem I'm walking out of that interview with a very negative opinion of your company.

Re: We analyzed thousands of interviews on everything from language to code style

#100
post #5

> Furthermore, no matter what, poor technical ability seems highly correlated with poor communication ability – regardless of language, it’s relatively rare for candidates to perform well technically but not effectively communicate what they’re doing (or vice versa), largely (and fortunately) debunking the myth of the incoherent, fast-talking, awkward engineer. My interpretation of this is that interviewees who can c…

Its fairly easy to see how this correlation breaks down. It is almost a truism that someone who is good at communicating with a wide range of people, is likely to make that wide range of people think they are more technically competent than people who aren't good at communicating with a wide range of people. Especially given the virtually random nature who you will communicate with during any given interview cycle.

But, its quite possible for people who are not naturally able to communicate with a wide range of people to be able to communicate with a smaller subset (say people they are more comfortable with, or people working on similar problems to them).

The more interesting question is, could those people who communicate less broadly, in a situation where they do communicate well, perform technically at the same level (or better) as people who are more natural communicators?

Another way to say this is, evaluating interviews is fairly prone to bias towards people who interview well. As someone who is hiring I'd be much more enthused about their results if it correlated to job performance, not hiring results. Its almost gospel at this point that there are people in software who interview well who do not make good hires, but the methodology of this article doesn't talk about this at all.

All told, nothing about this research is persuasive to me on my opinion of "interviews are largely worthless in determining who will be a good hire".

Post reply on HN