Live data from Hacker News

Technical interview performance is kind of arbitrary

blog.interviewing.io

51–60 of 246 posts

Re: Technical interview performance is kind of arbitrary

#51

With programmers, the single easiest way to identify good candidates (in my experience) is sheer interest in what they do / desire to learn. This is a learn everyday field and if you're interested in what you're doing, you're going to do a lot better at it. It's hard to apply yourself mentally to something that you don't have a good level of interest in. Given that it's a learn everyday field, people with that level…

As somebody who's terrible at monologuing, this scares me.

Re: Technical interview performance is kind of arbitrary

#52
Constantly confused by this, so much arguing back and forth, and yet the single easiest way to deal with this is a stripped down real world problem and then giving it to a whole bunch of different candidates. Some people adopt this some people don't some people argue that it's meaningless, and go back to the standard silly interview patterns of algorithm questions and meaningless complex fizzbuzz alternatives.

tptacek summarized this in his hiring post:

http://sockpuppet.org/blog/2015/03/06/the-hiring-post/

My personal conclusion is that most companies don't want this for two reasons:

1. culture fit is more important for people in a rigid hierarchical structure, partly because an out of the box thinker could be dangerous for that structure. too much questioning authority, too much pointing out flaws. It's much easier to have a good worker bee than wondering why you need 40 employees to build an automated gif platform.

2. in most companies everyone is very reluctant to make decisions. for example management struggles with clear direction because it opens them up to the question of liability. if they make a decision and it's wrong they might get fired. HR works the same way, if HR passes a resume along they want it to hit a list of keywords, so they can cover their asses if he turns out to be a bad hire.

Basically everyone is so scared to make a mistake that they make a lot more mistakes trying to avoid them.

The opening of the cracking the coding interview she talks about how they don't really care about false positives and negatives, they just want those to stay below a certain threshold. But consider the hiring scale of google compared to a small company and suddenly those things matter.

One bad hire can be toxic. And basing your hiring strategy on something a huge behemoth with infinite money does is kind of silly imho

Re: Technical interview performance is kind of arbitrary

#53
post #48
post #29

Take home tests FTW. The thinking goes as follows: 1. If the candidate can't be bothered to complete a 2-4 hour (depending on claimed seniority) code test in the language of their choice, we can't be bothered to talk to them. 2. If the candidate does reasonably well by completing the code test somewhat on time (with a fat margin allowed for them, well, having a life) and within parameters of the task, they're invited…

> 1. If the candidate can't be bothered to complete a 2-4 hour (depending on claimed seniority) code test in the language of their choice, we can't be bothered to talk to them. The approach you describe strikes me as shortsighted as well as to no small degree selfish. I don't do take-home tests, as a rule, for one overriding reason: my time is too valuable, and I have so much less of it to expend in discretionary fas…

How do you feel about the all day interview process some large companies are currently using? Keep in mind you probably had 1-3 phone + computer interviews before the in person interview.

A take home test seems like a less time intensive process as you at least don't need to take time off from your normal job to do it.

Re: Technical interview performance is kind of arbitrary

#54

Earlier quoted context omitted.

Interest is not enough, unfortunately. There are plenty of engineers who are attracted to the challenge and excitement of building new things, but have no appreciation for The Right Way to build things. Great engineers think beyond "how" and ask the "should" questions as well. Mediocre engineers glue things together in a haphazard way with little thought about what's the best way to write things. Caring about maintai…

Attention to detail will usually preclude someone from gluing things together like you describe though. Those are important details to account for and the type of person you're describing is more task focussed (aka - I got it done) than detail focussed (aka - I got it done right).

I would say this dichotomy doesn't quite capture what's being discussed. To my mind, detail focused can still result in an ugly hack of a solution, merely with all corner cases covered.

I think one possible phrase closer to the heart of things would be "aesthetically focused" as well as detail focused -- they care not only about the details of the functionality, but about the "technical beauty" of their solution.

Re: Technical interview performance is kind of arbitrary

#55
post #10

Earlier quoted context omitted.

I wonder how many employ Kahneman's recommendation based on his book, "Thinking, Fast and Slow": > Suppose that you need to hire a sales representative for your firm. If you are serious about hiring the best possible person for the job, this is what you should do. First, select a few traits that are prerequisites for success in this position (technical proficiency, engaging personality, reliability, and so on. Don't…

I like that. But just to be a little bit contrarian for the sake of conversation: Has it been shown that ignoring your gut instinct or even who you like most if inherently wrong? That kind of testing definitely removes a lot of bias, but it just assumes that biases are inherently bad. There certainly ARE bad biases (sexism, racism, etc). But you may also need to work with the individual you're recruiting, so how well…

Your question is one I've been exploring since the New Year.

One thing to consider is that the gut is a collection of relationships and groupings based on experiences and cognitive processes.

This is great for some circumstances. I don't need to take a tally of all the stats to ensure I'm making the most educated choice (based on reviews, calories, health safety, longitudinal studies, etc.) of which sandwich to buy at the deli.

On the flipside, our gut can betray us. One of the chapters of Malcolm Gladwell's Blink talks about a story of New York Cops whose instant reaction caused the wrongful death of a innocent because their heuristics that formed assumption after assumption were just flat out imprecise. Signs that should've flagged them to check their assumptions actually only reinforced their view. From their perspective they truly believed their conduct was in line with what they expected. In Thinking, Fast and Slow, Kahneman provides numerous gambles where our intuition guides us to make suboptimal or imprecise choices due to loss-aversion, endowment theory, et al.

I have to go but I don't have a perfect answer for you. But I will provide a Kindle two highlights that I highlighted in a book I'm reading called Rational Choice in an Uncertain World:

> Our decision-making capacities are not simply “wired in,” following some evolutionary design. Choosing wisely is a learned skill, which, like any other skill, can be improved with experience. An analogy can be drawn with swimming. When most of us enter the water for the first time, we do so with a set of muscular skills that we use to keep ourselves from drowning. We also have one important bias: We want to keep our heads above water. That bias leads us to assume a vertical position, which is one of the few possible ways to drown. Even if we know better, in moments of panic or confusion we attempt to keep our heads wholly free of the water, despite the obvious effort involved compared with that of lying flat in a “jellyfish float.” The first step in helping people learn to swim, therefore, is to make them feel comfortable with their head under water. Anybody who has managed to overcome the head-up bias can survive for hours by simply lying face forward on the water with arms and legs dangling—and lifting the head only when it is necessary to breathe (provided, of course, the waves are not too strong or the water too cold). Ordinary skills can thus be modified to cope effectively with the situation by removing a pernicious bias.

> The greatest obstacle to using external aids, such as the ones we will illustrate in this chapter, is the difficulty of convincing ourselves that we should take precautions against ourselves as Ulysses did. The idea that a self-imposed external constraint on action can actually enhance our freedom by releasing us from predictable and undesirable internal constraints is not an obvious one. It is hard to be Ulysses. The idea that such internal constraints can be cognitive, as well as emotional, is even less palatable. Thus, to allow our judgment to be constrained by the “mere numbers” or pictures or external aids offered by computer printouts is anathema to many people. In fact, there is even evidence that when such aids are offered, many experts attempt intuitively to improve upon these aids’ predictions—and then they do worse than they would have had they “mindlessly” adhered to them. Estimating likelihood does in fact involve mere numbers, but as Paul Meehl (1986) pointed out, “When you come out of a supermarket, you don’t eyeball a heap of purchases and say to the clerk, ‘Well, it looks to me as if it’s about $17.00 worth; what do you think?’ No, you add it up” (p. 372). Adding, keeping track, and writing down the rules of probabilistic inference explicitly are of great help in overcoming the systematic errors introduced by representative thinking, availability, anchor-and-adjust, and other biases. If we do so, we might even be able to learn a little bit from experience.

Re: Technical interview performance is kind of arbitrary

#56
post #49

Earlier quoted context omitted.

I assume the take home test is after you have talked to them? I wouldn't spend multiple hours unless it was already clear there was some desire to move further

This kind of depends on where the candidate is coming from. Those who're applying directly are obviously already expressing some desire to work with us. Recruiters should be instructed to give a consistent narrative, too. Good recruiters will also pitch the code test as a sign that we're a serious employer and take people seriously. I don't like phone conversations that serve as the first line of candidate triage. Ma…

A 4h test when your still rejecting 90+% of people is ridiculous. You are basically rejecting anyone that is talented ahead of time.

I personally know two programmers that did not create a resume for 20 years and still got jobs offers. Remember anything more complex than a phone screen is going to start turning people away.

Re: Technical interview performance is kind of arbitrary

#57

The headline is "interview performance is kind of arbitrary," but the data solution proposed in the article is "interviewers rate interviewees in a few different dimensions," which is not any less arbitrary. I appreciate there is an appendix addressing this issue, but it does not absolve the issues the analysis, especially since the appendix uses a "Versus Rating" to justify the statistical accuracy of the system, wh…

Thanks, as always, for the thoughtful notes. We certainly don't mean to imply that this is gospel, and there are limitations to the work, and fwiw, we are thinking about potentially moving to a 5 star scale.

That said, if click on the link in the footnotes to see the original data, you can get an idea of what we're working with.

And lastly, the fontawesome thingy isn't plotly. It was built using http://blockbuilder.org/ by @enjalot

Re: Technical interview performance is kind of arbitrary

#58
post #24

I had a really bizarre interview recently where, after the initial recruiter phone screen, I was rejected based on an in-person half-hour very simplistic paired coding exercise, only met with one person, and wasn't asked about my (imo very strong) resume once. I must have said something foolish at some point, which is on me, but the point is: interviews can be hit or miss. Fortunately you only need one hit.

Interesting. I wonder if we interviewed at the same place! I had a very similar experience over a year ago. At least I was told they only select one person from the group to move forward from the pair-programming task, which I thought was weird, but hey, it's their company.

Re: Technical interview performance is kind of arbitrary

#59
Great post and awesome interactive graph! props

I'm curious about the interviewer community. Specifically things like how are they vetted and how often they come to conduct interviews. It would be cool if there was a community of interviewers for the betterment of the process, but I could see their retention for conducting interviews to only be 1 or 2 before they drop out. I see in the appendix that there are those that do more, but no indication about what percent leave quickly.

A better drinking game might be when a candidate offers a data structure they know nothing about. Would a red-black tree work here? No.. I guess not.

Re: Technical interview performance is kind of arbitrary

#60
post #26

Earlier quoted context omitted.

most other job interview processes don't involve solving a series of riddles

What do they involve? How do you interview a sales guy? what do you ask him? How do you interview a product marketer?

I was a sales guy, I typically showed up with 6 months of pay stubs, dressed nicely, and had a job within hours.
Post reply on HN