Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making software engineering interviews predictive of job performance

qualified.io

Post reply on HN
Print view View original

51–60 of 113 posts

Re: Making software engineering interviews predictive of job performance

#51

Post by opportune »

derefr wrote: post #33
tracker1 wrote: post #4

In the end, it's a crap shoot... Even their highest marker of validity is only about 50%. In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. Where I work now, we have a small pre-interview code challenge..…

> In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. I mean, you're talking about a different thing than "job performance" here. Job performance is a satisficing measure: you only need to get someone who's…

Part of the idea of a universal standard in bigcorp is so that you can shuffle engineers between projects as they want to and as your strategy/needs change over time. And for cohesion.

Also I think there is a huge benefit for having generally capable employees, even if they are not directly utilizing many of their talents, and not counting the fungibility: they will make generally better decisions when reasoning about things outside their specific work domain and do better in ambiguous, not perfectly-specced situations. And in software at least, having general/complete knowledge and problem solving ability is absolutely vital if you want to be self-sufficient in diagnosing and solving problems, IMO.

Re: Making software engineering interviews predictive of job performance

#52

Post by RichardCA »

derefr wrote: post #33
tracker1 wrote: post #4

In the end, it's a crap shoot... Even their highest marker of validity is only about 50%. In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. Where I work now, we have a small pre-interview code challenge..…

> In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. I mean, you're talking about a different thing than "job performance" here. Job performance is a satisficing measure: you only need to get someone who's…

I can't help but object to statements that begin with "The point of a job interview is..."

It seems like we tend to lose the basic reality of gatekeeping and social ritual in these discussions.

It's never going to be empirical. It's how you strike the balance between the empirical part and the social ritual part, and no one ever wants to discuss that.

But I've certainly sat across the table from hiring managers who I knew weren't going to choose me based on a cursory glance, or just seemed like deeply frightened people.

Re: Making software engineering interviews predictive of job performance

#53

Post by jedmeyers »

dominotw wrote: post #40
jedmeyers wrote: post #13

Earlier quoted context omitted.

Not allowed by law.

> IQ tests (for interviewing) are typically spatial comparisons, some math, logic puzzles, reading comprehension, and pattern recognition I definitely got these in tech interviews. Thoughtworks gave me one last year.

It might have been a de facto IQ test, but they should have disguised it carefully.

Re: Making software engineering interviews predictive of job performance

#54

Post by tschiller »

thedance wrote: post #49
tschiller wrote: post #46

Earlier quoted context omitted.

If you know what kind of company/situation you're in, you can certainly select for candidates that are more likely to thrive in that environment. As you mentioned, people gravitate toward their strong suits. So by asking behavioral questions, you can learn how they approached situations in the past and whether their strengths/weaknesses are the right mix for your needs.

If your company knows what situation it is in, and if that situation persists for more than a year, your company is already better than many. The company I just left had an endless parade of new missions, new "values", new career ladder descriptions, and the median tenure of managers in engineering was about 6 months. The idea that you can either rate the performance of your staff or predictively hire engineers who w…

Yep, if your company is having leadership/strategy problems you're not going to go far.

But as a hiring manager in that situation, you can select for team members who excel at dealing with ambiguity, internal locus of control, vision, etc. Additionally, you would look for people who value learning vs. psychological safety, etc.

You're in a known situation, and that situation is constant change and churn

Re: Making software engineering interviews predictive of job performance

#55

Post by geoffroberts »

jamespollack wrote: post #26

so you're gonna pay us to do these "work samples" at market rate, right?

Actually, yes! Erik Bernhardsson, the CTO at Better.com who is cited in the article a few times, does exactly that for finalist candidates that they ask to provide a significant work sample. I think that's 100% appropriate. But that said, a work sample doesn't necessarily have to be something that takes hours on end either.

Re: Making software engineering interviews predictive of job performance

#56

Post by geoffroberts »

saviorand wrote: post #44

tl;dr: offer your applicants test exercises

Yes, but not generic tests—in order to design a hiring process that's actually predictive of future job performance, the "tests" have to be designed to be directly relevant to the real world work that the engineer will be tasked with.

Re: Making software engineering interviews predictive of job performance

#57

Post by geoffroberts »

tracker1 wrote: post #4

In the end, it's a crap shoot... Even their highest marker of validity is only about 50%. In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. Where I work now, we have a small pre-interview code challenge..…

I completely agree that if you're designing an sort of assessment process, you should go through it yourself or ask other engineers to go through it so it's reasonably scoped in terms of expected time commitment. You can also use assessment tools that specifically limit how much time an engineer can invest in working on the challenge, which is also useful in showing how much progress each candidate made in a set amount of time. The problem with this approach can often be that the candidate feels like they're working against the clock though, which isn't "real world."

Re: Making software engineering interviews predictive of job performance

#58

Post by geoffroberts »

RichardCA wrote: post #52
derefr wrote: post #33

Earlier quoted context omitted.

> In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. I mean, you're talking about a different thing than "job performance" here. Job performance is a satisficing measure: you only need to get someone who's…

I can't help but object to statements that begin with "The point of a job interview is..." It seems like we tend to lose the basic reality of gatekeeping and social ritual in these discussions. It's never going to be empirical. It's how you strike the balance between the empirical part and the social ritual part, and no one ever wants to discuss that. But I've certainly sat across the table from hiring managers who I…

Agreed that interviews are both a gatekeeping and social ritual, but from the perspective of the employer I do think "the point of a job interview" should very clearly be to identify candidates with the highest probability of being successful in the role.

Re: Making software engineering interviews predictive of job performance

#59

Post by geoffroberts »

tschiller wrote: post #21
MaxBarraclough wrote: post #12

Earlier quoted context omitted.

> if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel And where you set the bar will depend on, roughly speaking, your prestige. Google can afford to be extremely selective when hiring technologists. Similar thing goes for Harvard and the Air Force. > what a person works on (developers) outside of educa…

All teams should be selective. But if you don't have a Google-sized brand and budget, you need to develop a clearer understanding of your teams strengths/weaknesses and needs. This understanding will enable you to figure out which hiring criteria do you need to maximize vs. which do you need to satisfy and where the bar is. Also, by interviewing more effectively, you punch above your weight by spending more face-to-f…

I love this point—the more time you can spend selling the candidate the better. And I think the more effective your interview process, candidates take notice and probably need to be "sold" that much less.

Re: Making software engineering interviews predictive of job performance

#60

Post by vbezhenar »

lmkg wrote: post #27
amznthrowaway5 wrote: post #16

Earlier quoted context omitted.

Wouldn't coding interviews show the same bias?

The difference is, the Supreme Court has already made a ruling on IQ tests.

Isn't this ruling inherently racist?
Post reply on HN