Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making software engineering interviews predictive of job performance

qualified.io

Post reply on HN
Print view View original

61–70 of 113 posts

Re: Making software engineering interviews predictive of job performance

#61

Post by geoffroberts »

tschiller wrote: post #15
tracker1 wrote: post #4

In the end, it's a crap shoot... Even their highest marker of validity is only about 50%. In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. Where I work now, we have a small pre-interview code challenge..…

Great points - there are things we're trying to address over at Interview GPS. We're building a Wikipedia/Stack Overflow-style knowledge base of the best interviewer guides/rubrics: come join the cause at https://www.interviewgps.com/community/ We tag questions in 3 areas: Personal Values: instead of "culture" we help you assess shared values. For example: do you value fun vs. achievement? Competencies: does your sof…

Love your site—thanks for sharing! And +1 with regards to the potential legal issues.

Re: Making software engineering interviews predictive of job performance

#62

Post by vbezhenar »

bitwize wrote: post #14
amznthrowaway5 wrote: post #11

Earlier quoted context omitted.

Why not just give an actual IQ test?

That opens you up to liability for race-based discrimination. The accepted wisdom is that IQ tests are inherently racist and favor white evaluees over black and brown ones.

What about other countries?

Re: Making software engineering interviews predictive of job performance

#63

Post by geoffroberts »

Havoc wrote: post #39

I don't think you can. Job performance is something that gets figured out on the fly. Unless you're working in a crazy standardized operation thats a moving target. e.g. We've got a ton of manager & they all sorta do their own thing...they gravitate towards their strong suit essentially. You can't really test for that in advance. Some people excel at leading 30 man teams on stable jobs. Others excel at being parachut…

So you need to design you assessment process to reflect the real world scenarios the hire will encounter. It's surprising to me to hear so many developers push back on this notion that you can't design a predictive hiring process. If you were hiring a dentist, wouldn't a great way to assess them be having them fill a cavity? Or asking a chef to make you your restaurant's signature recipe? It seems very logical and the key here is that the work sample is reflective of what they'll encounter on the job.

Re: Making software engineering interviews predictive of job performance

#64

Post by geoffroberts »

thedance wrote: post #49
tschiller wrote: post #46

Earlier quoted context omitted.

If you know what kind of company/situation you're in, you can certainly select for candidates that are more likely to thrive in that environment. As you mentioned, people gravitate toward their strong suits. So by asking behavioral questions, you can learn how they approached situations in the past and whether their strengths/weaknesses are the right mix for your needs.

If your company knows what situation it is in, and if that situation persists for more than a year, your company is already better than many. The company I just left had an endless parade of new missions, new "values", new career ladder descriptions, and the median tenure of managers in engineering was about 6 months. The idea that you can either rate the performance of your staff or predictively hire engineers who w…

That's true.... that might not be a company you want to work for. If there's not clear strategic direction and things constantly shift, it's very hard to hire anybody effectively into the org.

Re: Making software engineering interviews predictive of job performance

#65

Post by geoffroberts »

thedance wrote: post #48
Havoc wrote: post #39

I don't think you can. Job performance is something that gets figured out on the fly. Unless you're working in a crazy standardized operation thats a moving target. e.g. We've got a ton of manager & they all sorta do their own thing...they gravitate towards their strong suit essentially. You can't really test for that in advance. Some people excel at leading 30 man teams on stable jobs. Others excel at being parachut…

This whole article is predicated on the idea that you can quantitatively measure the performance of a software engineer. That seems completely unfounded.

I don't think that true—what the article is saying is that different hiring activities are more or less useful in predicting the future success of a hire. And the more measures you employee intelligently, the more predictive the process becomes.

Re: Making software engineering interviews predictive of job performance

#66

Post by geoffroberts »

driverdan wrote: post #47
new_realist wrote: post #3

“General Mental Ability (GMA) tests (like the IQ test) are very predictive of future job performance, largely because people with high intelligence can learn the skills needed to be successful on the job more rapidly. However, due to legal concerns they’re not recommended for companies hiring in the US.” This is what a classic FAANG interview is: an IQ test which is disguised as a relevant skills test for legal reaso…

I'd rather have a hard worker with an average IQ than someone who's lazy with a high IQ. Besides, IQ tests have biases that don't translate to good job performance. For example, if English isn't your first language you may do worse on an English-based IQ test.

Agreed. Ambition is an intangible that should also be assessed during the hiring process.

Re: Making software engineering interviews predictive of job performance

#67

Post by geoffroberts »

kube-system wrote: post #23

Some of the points here are good, but I think some of them are too simplistic. This article makes a huge assumption: that performance is the only factor you're hiring for, and therefore unstructured interviews are worthless. That would make sense if you're interviewing for a factory of emotionless robots. But humans are social beings, and human performance as a team is more complicated than a sum of individual produc…

Great points—I completely agree that the human element of hiring is of critical importance when it comes to building effective and high functioning teams. This article is intended to focus on the more technical/job focused component of hiring. That said, I agree with your comments here across the board.

Re: Making software engineering interviews predictive of job performance

#68

Post by jhoffner »

thedance wrote: post #48
Havoc wrote: post #39

I don't think you can. Job performance is something that gets figured out on the fly. Unless you're working in a crazy standardized operation thats a moving target. e.g. We've got a ton of manager & they all sorta do their own thing...they gravitate towards their strong suit essentially. You can't really test for that in advance. Some people excel at leading 30 man teams on stable jobs. Others excel at being parachut…

This whole article is predicated on the idea that you can quantitatively measure the performance of a software engineer. That seems completely unfounded.

Why isn't it? Any job where enough information can be known to know that someone should be fired or promoted is a job that can be quantitatively measured to some degree.

Re: Making software engineering interviews predictive of job performance

#69

Post by OrangeMango »

wpietri wrote: post #41
derefr wrote: post #33

Earlier quoted context omitted.

> In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. I mean, you're talking about a different thing than "job performance" here. Job performance is a satisficing measure: you only need to get someone who's…

> Job performance is a satisficing measure: you only need to get someone who's good enough to perform the duties you need of them. (Yes, even in software engineering.) Is that really the case for software? I mean, I'm very skeptical of the "only the best and brightest" nonsense, as I think it selects for the wrong people. But I'm also skeptical of the "eh, good enough" heuristic. Right now I'm helping a friend deal w…

> Now that I can see the code, I understand why the dev shop needed so many people working on it, and why all features were hard to do. It's because when you use a lot of mediocre programmers, they all create work for themselves and one another. Not intentionally, but just because they aren't able to see how the differing approaches and sloppy work slowly diminish productivity.

To me, that sounds like a leadership problem at the outside dev shop. Properly supervised, properly supported, and with reasonable deadlines, mediocre developers can provide extraordinary results per dollar spent on probably 80-90% of dev tasks.

Re: Making software engineering interviews predictive of job performance

#70

Post by geoffroberts »

hn_throwaway_99 wrote: post #45

2 comments on this: 1. The authors of this paper are selling something. This is an advertisement. Not saying it's wrong or untruthful because of that, but it is an advertisement. 2. As another commenter said, the cited data is from a paper that doesn't look at software engineers specifically. Given that software engineering has one of the highest amounts of variability in productivity of any profession, that should b…

The purpose of this paper is to present research that's completely independent of any product, and show how the concepts apply to software engineering specifically. What's your source in saying "software engineering has one of the highest amounts of variability in productivity of any profession?" That seems hugely unfounded.
Post reply on HN