We analyzed thousands of interviews on everything from language to code style • Hacker News Board

Re: We analyzed thousands of interviews on everything from language to code style

#161

Post by notyourloops » Thu, Jun 15, 2017, 1:25 PM UTC

lettergram wrote: post #14

Jemaclus wrote: post #5

> Furthermore, no matter what, poor technical ability seems highly correlated with poor communication ability – regardless of language, it’s relatively rare for candidates to perform well technically but not effectively communicate what they’re doing (or vice versa), largely (and fortunately) debunking the myth of the incoherent, fast-talking, awkward engineer. My interpretation of this is that interviewees who can c…

That's actually how I conduct interviews. It's been immensely successful at weeding out candidates within 5 minutes; although most interviews are 45 - 60 minutes. Basically, give them less than ten lines of code, ask them what it does, where are a couple bugs, ask what would you name the function, etc. Then we talk about how to improve it. I'd say, less than 20% of people I interview pass. It's actually amazingly low…

That's great! I really hope interviews like that become more common.

Re: We analyzed thousands of interviews on everything from language to code style

#162

Post by SonOfLilit » Thu, Jun 15, 2017, 1:55 PM UTC

tptacek wrote: post #152

One of the language results doesn't make sense. It claims that it matters, significantly, if you solve interview problems in Java when the hiring company is a Java shop, but not when the hiring company is a C++ shop. But that's reversed. It is in fact fairly difficult for a high-level language programmer to pick up C++, and facility with C++ (or at least C) is a common, accepted goal for C++ hiring shops. A C++ shop…

They give P ~= 0.36 to this claim, in an article that I think makes more than 1/0.36 ~= 28 claims, so I would suspect that maybe that's just a coincidence in their data?

Re: We analyzed thousands of interviews on everything from language to code style

#163

Post by zebraflask » Thu, Jun 15, 2017, 2:04 PM UTC

graphememes wrote: post #95

zebraflask wrote: post #92

A lot of this article rings true with my experience. And I agree with the comments dinging live coding tests - those are the worst. I can't code effectively unless I'm calm and can concentrate, and these things are almost designed to get you off-balance. Even more galling when you have a healthy GitHub portfolio that they refuse to even look at in favor of a quiz (this has happened recently).

To note a few things regarding the interview and hiring process (I do a lot of it, have done about 20 in the past two days), the reason for the quiz and not using your portfolio (unless its extremely outstanding and you are fine doing a live coding questionnaire) is so that we can judge all the candidates equally and fairly. It gives a common baseline to judge. Each candidate does the same thing and we have a good id…

I agree with you about irrelevant questions, and out of the dozen or so coding quizzes I've done over the past several years, mostly with start up companies, those are unfortunately usually what I got. One company was advertising for a front end role and, sensibly enough, asked front end-related questions - that was the exception.

And the thought that quizzes provide a fair point of comparison comes across to me as putting process ahead of substance. Interviewing isn't meant to be fair to everyone - only one person gets the job, after all - so it's not like handing out cookies and stickers in middle school. It's meant to see, in part, whether the person is capable of generating working code. If you have a person who can provide samples to prove it, requiring an artificial quiz really is a slap in the face to a lot of good candidates.

Re: We analyzed thousands of interviews on everything from language to code style

#164

Post by bluGill » Thu, Jun 15, 2017, 2:56 PM UTC

mmcnl wrote: post #86

bluGill wrote: post #58

Earlier quoted context omitted.

No, it would not be a perfect answer, and could work against you: it shows you are not willing to work with me and assume some intelligence on my part (until proven otherwise) and thus you are potentially not going to work well with the rest of the team. I would respond that we have already profiled this code and discovered it is a bottleneck, now answer the question. It won't be hard to overcome the negative impress…

I fail to see why analyzing a problem before solving it somehow translates into "not going to work well with the team". Problem analysis skills are far more valuable than problem solving skills. Anyone can search on Google for a solution to a given problem, but no one on Google can tell you what the actual problem is. If you really thing the answer I was responding to is not acceptable, then the question is a lousy q…

Remember, this is an interview situation, I can't give you a realistic 500000 line program with a slow hot spot and ask you to fix: we only have a few minutes (maybe an hour). I have simplified the problem to a single function that is a bottleneck, I want you to analyze this function so I can see how well you can analyze things.

Refusing to do something because it hasn't been profiled is not analysis, it is refusing to do analysis. You have been given a problem and been asked to analyze it. That refusal makes you not a team player.

Once again, this is an interview. Noting that this scenario (a single function that is known to be too slow for reasons that can be fixed in 10 minutes) never happens in the real world is good - it shows you understand that real world analysis includes elements not included here. Stating you won't do this because you don't have profile data is not being a team player.

Re: We analyzed thousands of interviews on everything from language to code style

#165

Post by DoofusOfDeath » Thu, Jun 15, 2017, 3:39 PM UTC

bluGill wrote: post #164

mmcnl wrote: post #86

Earlier quoted context omitted.

I fail to see why analyzing a problem before solving it somehow translates into "not going to work well with the team". Problem analysis skills are far more valuable than problem solving skills. Anyone can search on Google for a solution to a given problem, but no one on Google can tell you what the actual problem is. If you really thing the answer I was responding to is not acceptable, then the question is a lousy q…

Remember, this is an interview situation, I can't give you a realistic 500000 line program with a slow hot spot and ask you to fix: we only have a few minutes (maybe an hour). I have simplified the problem to a single function that is a bottleneck, I want you to analyze this function so I can see how well you can analyze things. Refusing to do something because it hasn't been profiled is not analysis, it is refusing…

I think you misconstrued the intent of my original post. I was merely trying to point out that the interview question was (in my experience) based on a very questionable premise.

I wasn't saying anything about how one ought to handle such a situation during an interview.

I totally agree that a response akin to "Your question makes wrong assumptions. Let's move on to the next question." would be completely inappropriate for an interview, or if the same issue arose in a normal work situation.

Re: We analyzed thousands of interviews on everything from language to code style

#166

Post by jroseattle » Thu, Jun 15, 2017, 4:18 PM UTC

watwut wrote: post #140

jroseattle wrote: post #125

Earlier quoted context omitted.

This is significant, and can be more so depending on the company and environment. I've had my own team members tell me they aren't engaged, don't speak up in meetings, etc. because they are an introvert. Like it's a condition we must accommodate. Bullshit. Communication is a skill. We value clear communication in our company. If you join our company, you may not be an expert in communication, but you will grow that s…

"don't speak up in meetings" People should speak in meetings when they have something to say. Not just so they speak - that just waste everybody time. The more important point is that if 80% of people shut down while they are talking to someone, maybe has more to do with that someone then 80% of population being uncommunicative.

We're not quite that naive and clumsy. I'm the first one to make sure we keep discussion on-point and actively shutdown the monologuers of our team.

So let's clarify -- "speak up in a meeting" means when a senior person doesn't provide input at a time that would provide better direction, identify issues earlier in a process, etc. If you're a senior engineer, I need your input to ensure we're doing the right thing(s).

The forum of a group meeting isn't always in the wheelhouse of someone who freezes up when speaking in front of others. It's understandable -- I used to be one of those people. It can be out of your comfort zone, and doing so feels super-risky as well as plain frightening. It is still just a skill to be learned, no different than building muscles through exercise.

In our group, we talk a lot about trust and support. Everything we do is in the spirit of making each other stronger. Everyone has strengths and weaknesses -- share your strengths with the group, and build your weaknesses from the strength of others. It sounds pie-in-the-sky, but as the head of my group, I make sure we take this seriously.

So we don't have 80% of our group shutting down -- nobody talks over the top of anyone else, and we don't allow it. We need the input of the best of those, and sometimes that involves speaking up. If you're on my team and that notion makes you nervous, I understand. But as I've always explained to my team, we're going to help you build skills, and communicating is one of them.

Re: We analyzed thousands of interviews on everything from language to code style

#167

Post by 11thEarlOfMar » Thu, Jun 15, 2017, 4:21 PM UTC

lj3 wrote: post #110

11thEarlOfMar wrote: post #89

Earlier quoted context omitted.

We do it in reverse. In the first 2 hour interview, candidates learn our tech from 2 or 3 engineers. We then send them home with a set of coding tasks that are all relevant to what we do. As in functions of utilities that we actually would need to write, as opposed to puzzles. They choose one of 6 tasks, write ~100 lines of code, then come back for the next 2 hour interview and lead a code review of the code they wro…

So, based on the new information presented in the article that people who can talk clearly about code can also write code, are you likely to change your interview practices to omit the take home task and ask the candidate to talk about code that somebody else wrote? It would be faster, easier and according to the data in TFA just as accurate, yes?

Do you have data that shows how fast and easy the take home test approach is so I can compare? I only have anecdotal data.

I do give candidates the option of the take home vs. white boarding. So far (it's admittedly still a pretty small sample) they've all opted for the take home. However, we'll take a look at this approach as well.

Re: We analyzed thousands of interviews on everything from language to code style

#168

Post by jroseattle » Thu, Jun 15, 2017, 4:27 PM UTC

wolco wrote: post #133

jroseattle wrote: post #125

Earlier quoted context omitted.

This is significant, and can be more so depending on the company and environment. I've had my own team members tell me they aren't engaged, don't speak up in meetings, etc. because they are an introvert. Like it's a condition we must accommodate. Bullshit. Communication is a skill. We value clear communication in our company. If you join our company, you may not be an expert in communication, but you will grow that s…

If you want diversity you try to accommodate different personaility types. Otherwise you end up with a mono culture. There is value in silence and limited communication.

Communication is a behavior and act, not a personality type. The suggestion here is that it's ok for someone who doesn't communicate well to be accommodated in the name of diversity. The notion of "poor communicator" as a personality type doesn't jive with me.

Replace "communication" with "data modeling" or "project management" or "reading". There are plenty who don't necessarily possess those skills.

Should we expect that others shouldn't develop skills in those areas (assuming they are important for their role) in the name of "diversity"?

Re: We analyzed thousands of interviews on everything from language to code style

#169

Post by SilasX » Thu, Jun 15, 2017, 5:33 PM UTC

bluGill wrote: post #164

mmcnl wrote: post #86

Earlier quoted context omitted.

I fail to see why analyzing a problem before solving it somehow translates into "not going to work well with the team". Problem analysis skills are far more valuable than problem solving skills. Anyone can search on Google for a solution to a given problem, but no one on Google can tell you what the actual problem is. If you really thing the answer I was responding to is not acceptable, then the question is a lousy q…

Remember, this is an interview situation, I can't give you a realistic 500000 line program with a slow hot spot and ask you to fix: we only have a few minutes (maybe an hour). I have simplified the problem to a single function that is a bottleneck, I want you to analyze this function so I can see how well you can analyze things. Refusing to do something because it hasn't been profiled is not analysis, it is refusing…

That's the problem -- you're smuggling in non-obvious, hidden assumptions about the framing of the interview[1] and then penalizing someone who doesn't share them, without giving them the chance to isolate what those assumptions were and why they should use those in this context.

So they're getting rejected for a reason other than technical skill.

[1] "Okay, we're going to talk like I've already fixed problems A, B, and C, but not X, Y, and Z, because we obviously don't have time for a full interaction that is open-ended enough to allow for all of those to be an issue, and you're being judged on how quickly you focus on and solve the X/Y/Z."

Re: We analyzed thousands of interviews on everything from language to code style

#170

Post by dando » Thu, Jun 15, 2017, 5:51 PM UTC

lettergram wrote: post #14

Jemaclus wrote: post #5

> Furthermore, no matter what, poor technical ability seems highly correlated with poor communication ability – regardless of language, it’s relatively rare for candidates to perform well technically but not effectively communicate what they’re doing (or vice versa), largely (and fortunately) debunking the myth of the incoherent, fast-talking, awkward engineer. My interpretation of this is that interviewees who can c…

That's actually how I conduct interviews. It's been immensely successful at weeding out candidates within 5 minutes; although most interviews are 45 - 60 minutes. Basically, give them less than ten lines of code, ask them what it does, where are a couple bugs, ask what would you name the function, etc. Then we talk about how to improve it. I'd say, less than 20% of people I interview pass. It's actually amazingly low…

When you put people under the pressure of not having a job, a time limit and worst, the pressure of having someone looking over you, bugging you, and judging you, it can be incredibly hard to think straight or say anything at all.

You can act buddy all you want, but people aren't stupid. They know if they take a few extra seconds too long, you're already thinking "this guy sucks"