Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making software engineering interviews predictive of job performance

qualified.io

Post reply on HN
Print view View original

91–100 of 113 posts

Re: Making software engineering interviews predictive of job performance

#91

Post by pmiller2 »

tracker1 wrote: post #4

In the end, it's a crap shoot... Even their highest marker of validity is only about 50%. In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. Where I work now, we have a small pre-interview code challenge..…

> In the end, it's a crap shoot... Even their highest marker of validity is only about 50%.

Here's the thing though: all these different tests aren't completely independent. You can combine a couple of them and get a criterion validity value of up to .65.

See https://books.google.com/books?id=IvKYhrXAfRUC&pg=PA198&lpg=...

Re: Making software engineering interviews predictive of job performance

#93

Post by LyndsySimon »

driverdan wrote: post #47
new_realist wrote: post #3

“General Mental Ability (GMA) tests (like the IQ test) are very predictive of future job performance, largely because people with high intelligence can learn the skills needed to be successful on the job more rapidly. However, due to legal concerns they’re not recommended for companies hiring in the US.” This is what a classic FAANG interview is: an IQ test which is disguised as a relevant skills test for legal reaso…

I'd rather have a hard worker with an average IQ than someone who's lazy with a high IQ. Besides, IQ tests have biases that don't translate to good job performance. For example, if English isn't your first language you may do worse on an English-based IQ test.

Have you by chance read anything by Kurt von Hammerstein-Equord? He’d strongly disagree with you:

> I distinguish four types. There are clever, hardworking, stupid, and lazy officers. Usually two characteristics are combined. Some are clever and hardworking; their place is the General Staff. The next ones are stupid and lazy; they make up 90 percent of every army and are suited to routine duties. Anyone who is both clever and lazy is qualified for the highest leadership duties, because he possesses the mental clarity and strength of nerve necessary for difficult decisions. One must beware of anyone who is both stupid and hardworking; he must not be entrusted with any responsibility because he will always only cause damage.

Re: Making software engineering interviews predictive of job performance

#94

Post by candybar »

kube-system wrote: post #23

Some of the points here are good, but I think some of them are too simplistic. This article makes a huge assumption: that performance is the only factor you're hiring for, and therefore unstructured interviews are worthless. That would make sense if you're interviewing for a factory of emotionless robots. But humans are social beings, and human performance as a team is more complicated than a sum of individual produc…

With respect to your point about unstructured interviews - keep in mind that any statistical measure of performance isn't based on an objective measure of output - it's almost certainly based on performance ratings, which take into account "impact on the entire team." If anything, my experience is that charming, likable people that do well in unstructured interviews are also better at getting better performance ratings than they deserve.

Also, I think you're implicitly assuming that structured interviews cannot measure soft skills as they relate to workplace behavior. I would argue the opposite - the only way to ensure high standards for these types of skills is to have a structured interview. Otherwise, you're not systematically trying to hire people whose workplace behavior enhances group productivity, you're just hoping that whatever random biases that interviewers bring to the table end up being a net positive. Being a productive team player isn't about being able to impress a random person in an unstructured conversation - it's about knowing how to behave productively (combined with the willingness to do so, which unfortunately is contextual and virtually impossible to directly observe) in a wide variety of challenging situations.

Re: Making software engineering interviews predictive of job performance

#95

Post by hinkley »

Havoc wrote: post #39

I don't think you can. Job performance is something that gets figured out on the fly. Unless you're working in a crazy standardized operation thats a moving target. e.g. We've got a ton of manager & they all sorta do their own thing...they gravitate towards their strong suit essentially. You can't really test for that in advance. Some people excel at leading 30 man teams on stable jobs. Others excel at being parachut…

If you hire all the exact same kind of people, what happens when you were wrong about the sort you need? A little diversity is good. We don’t need everyone to be an expert on everything, we just need a few experts for everything.

Re: Making software engineering interviews predictive of job performance

#96

Post by oreally »

LyndsySimon wrote: post #93
driverdan wrote: post #47

Earlier quoted context omitted.

I'd rather have a hard worker with an average IQ than someone who's lazy with a high IQ. Besides, IQ tests have biases that don't translate to good job performance. For example, if English isn't your first language you may do worse on an English-based IQ test.

Have you by chance read anything by Kurt von Hammerstein-Equord? He’d strongly disagree with you: > I distinguish four types. There are clever, hardworking, stupid, and lazy officers. Usually two characteristics are combined. Some are clever and hardworking; their place is the General Staff. The next ones are stupid and lazy; they make up 90 percent of every army and are suited to routine duties. Anyone who is both c…

This is a quote from one man's experience and you're going to treat it as though it's the answer to his concerns? really?

Re: Making software engineering interviews predictive of job performance

#97

Post by oreally »

iateanapple wrote: post #42
vsareto wrote: post #36

Earlier quoted context omitted.

IQ has nothing to do with whether you know CS basics or not, that's the problem. It doesn't matter that they said it's a disguised IQ test because it's not an IQ test in any fashion.

> IQ has nothing to do with whether you know CS basics or not, that's the problem. In practice the Google/Facebook interview has nothing to do with knowing the CS basics either. Edit: In practice understanding CS basics is the baseline that gets you to the interview - kind of like English is the baseline for an IQ test delivered in English.

> In practice the Google/Facebook interview has nothing to do with knowing the CS basics either.

What the heck is this statement? You do realize many algorithms that they test for are taught in an algorithms class for a Computer Science course?

I doubt you took an IQ test before or even properly accessed what it is. there are IQ tests composed of entirely of pattern matching shapes and they do not have any knowledge/language prerequisites. Get outta my internet.

Re: Making software engineering interviews predictive of job performance

#98

Post by oreally »

xenihn wrote: post #86
dominotw wrote: post #34

Earlier quoted context omitted.

you can pass every facebook iterview by practicing facebook tagged questions on leetcode. People do that every single day, here is an example from today https://leetcode.com/discuss/interview-question/551434/faceb... Its NOT related to IQ. You can search leetcode forum for tons of examples of actual FB questions, all of them are tagged on leetcode.

You can get better at IQ tests by practicing them.

What a loaded statement. 1) You're already assuming these FAANG tests are an IQ test when people are discussing whether they are. 2) You're taking an assumed characteristic that tests that can be practiced and that makes them IQ tests.

Practicing for these tests makes you a rote learner and and rote learners by definition have too much crystallized knowledge, and they fail when met with new situations, which is not what intelligence tries to define. FAANG tests hence are not IQ tests if you can practice them.

Essentially these assumptions that FAANG tests are disguised IQ tests are just a reflection of the ego of those who are perpetuating them.

Anyone who feels like replying to this out of emotion they because they're in FAANG or whatever should take a psychology/psychometrics class first (yes, something outside of the software sector) before replying.

Re: Making software engineering interviews predictive of job performance

#99

Post by closeparen »

derefr wrote: post #33
tracker1 wrote: post #4

In the end, it's a crap shoot... Even their highest marker of validity is only about 50%. In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. Where I work now, we have a small pre-interview code challenge..…

> In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. I mean, you're talking about a different thing than "job performance" here. Job performance is a satisficing measure: you only need to get someone who's…

Sometimes when I delegate it costs me more time explaining and supervising than it would to do the work myself.

Sometimes I get back a perfectly decent solution, but in 5x the time I would have considered reasonable.

Most often I get back something that’s not amazing but looks fine, and works at first but turns out to have had some sloppiness or impedance mismatch down the road.

Sometimes I get back not only a solid and thorough solution, but a coworker who went way deeper into the topic than was necessary for the particular task, and is excitedly teaching me new things about it.

From certain people I get back an explanation of why what I’m asking will turn out to be a mistake 8 moves from now in the 10-dimensional chess game, and thing I actually needed instead.

The third group is technically “good enough.” They pull their weight, and we turn a profit on having them around. But if I can manage it, I’ll always try to get the fourth group for junior positions and the fifth group for senior (between those two it’s the same set of people at different stages of development). Those people are maybe 20% of my division, but they do 80% of the work, for what I expect is closer to 30% of the cost.

Re: Making software engineering interviews predictive of job performance

#100

Post by iateanapple »

oreally wrote: post #97
iateanapple wrote: post #42

Earlier quoted context omitted.

> IQ has nothing to do with whether you know CS basics or not, that's the problem. In practice the Google/Facebook interview has nothing to do with knowing the CS basics either. Edit: In practice understanding CS basics is the baseline that gets you to the interview - kind of like English is the baseline for an IQ test delivered in English.

> In practice the Google/Facebook interview has nothing to do with knowing the CS basics either. What the heck is this statement? You do realize many algorithms that they test for are taught in an algorithms class for a Computer Science course? I doubt you took an IQ test before or even properly accessed what it is. there are IQ tests composed of entirely of pattern matching shapes and they do not have any knowledge/…

> What the heck is this statement?

If you think knowledge of CS fundamentals is what is separating the hired from the rejected...

My experience from interviewing hundreds of candidates over the years at a top tech company is that the majority of candidates we reject have strong CS fundamentals.

> there are IQ tests composed of entirely of pattern matching shapes

And there are IQ tests/proxies that are entirely “verbal” like the GSS Wordsum - what is your point?

> I doubt you took an IQ test before or even properly accessed what it is.

I’m certified in my state to administer IQ tests....

Post reply on HN