Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Making software engineering interviews predictive of job performance

qualified.io

Post reply on HN
Print view View original

41–50 of 113 posts

Re: Making software engineering interviews predictive of job performance

#41

Post by wpietri »

derefr wrote: post #33
tracker1 wrote: post #4

In the end, it's a crap shoot... Even their highest marker of validity is only about 50%. In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. Where I work now, we have a small pre-interview code challenge..…

> In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. I mean, you're talking about a different thing than "job performance" here. Job performance is a satisficing measure: you only need to get someone who's…

> Job performance is a satisficing measure: you only need to get someone who's good enough to perform the duties you need of them. (Yes, even in software engineering.)

Is that really the case for software? I mean, I'm very skeptical of the "only the best and brightest" nonsense, as I think it selects for the wrong people. But I'm also skeptical of the "eh, good enough" heuristic.

Right now I'm helping a friend deal with a code base that was built by an outside dev shop. It undeniably works; the business has been running on it for a number of years. And honestly, like most software, it's not a complicated app; mostly CRUD stuff. But overall it's a spaghetti snarl. Any given concept might be in the code, or in a config file, or driven by SQL data, or even driven by JSON blobs stuffed into the database, mutated by the main code base, and passed along to the JS front end.

Now that I can see the code, I understand why the dev shop needed so many people working on it, and why all features were hard to do. It's because when you use a lot of mediocre programmers, they all create work for themselves and one another. Not intentionally, but just because they aren't able to see how the differing approaches and sloppy work slowly diminish productivity.

So although I agree with you that a lot of companies hire on vanity metrics, I'm really skeptical that "whoever, as long as they barely qualify" is a better approach.

Re: Making software engineering interviews predictive of job performance

#42

Post by iateanapple »

vsareto wrote: post #36
derefr wrote: post #35

Earlier quoted context omitted.

That's why the parent said that they're "disguised" IQ tests. If they used a test that strongly correlated with the outcome of an IQ test, then it would also , legally, be an IQ test, and they'd have to stop doing it. So instead, they use a test that weakly correlates with the outcome of an IQ test, and then try to extract the a measure of IQ from the "noise" that is all the other variables the test depends on.

IQ has nothing to do with whether you know CS basics or not, that's the problem. It doesn't matter that they said it's a disguised IQ test because it's not an IQ test in any fashion.

> IQ has nothing to do with whether you know CS basics or not, that's the problem.

In practice the Google/Facebook interview has nothing to do with knowing the CS basics either.

Edit: In practice understanding CS basics is the baseline that gets you to the interview - kind of like English is the baseline for an IQ test delivered in English.

Re: Making software engineering interviews predictive of job performance

#43

Post by iateanapple »

derefr wrote: post #33
tracker1 wrote: post #4

In the end, it's a crap shoot... Even their highest marker of validity is only about 50%. In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. Where I work now, we have a small pre-interview code challenge..…

> In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. I mean, you're talking about a different thing than "job performance" here. Job performance is a satisficing measure: you only need to get someone who's…

> Does it matter if someone is "the best" if they're still not good enough to do your job?

Except we are hiring for a mix of performance and potential - weighted towards potential.

I don’t want to hire a junior engineer that can never make it to senior.

When I am only hiring for performance I hire contractors - and the bar is a lot lower.

Re: Making software engineering interviews predictive of job performance

#45

Post by hn_throwaway_99 »

2 comments on this:

1. The authors of this paper are selling something. This is an advertisement. Not saying it's wrong or untruthful because of that, but it is an advertisement.

2. As another commenter said, the cited data is from a paper that doesn't look at software engineers specifically. Given that software engineering has one of the highest amounts of variability in productivity of any profession, that should be taken into close consideration.

Re: Making software engineering interviews predictive of job performance

#46

Post by tschiller »

Havoc wrote: post #39

I don't think you can. Job performance is something that gets figured out on the fly. Unless you're working in a crazy standardized operation thats a moving target. e.g. We've got a ton of manager & they all sorta do their own thing...they gravitate towards their strong suit essentially. You can't really test for that in advance. Some people excel at leading 30 man teams on stable jobs. Others excel at being parachut…

If you know what kind of company/situation you're in, you can certainly select for candidates that are more likely to thrive in that environment.

As you mentioned, people gravitate toward their strong suits. So by asking behavioral questions, you can learn how they approached situations in the past and whether their strengths/weaknesses are the right mix for your needs.

Re: Making software engineering interviews predictive of job performance

#47

Post by driverdan »

new_realist wrote: post #3

“General Mental Ability (GMA) tests (like the IQ test) are very predictive of future job performance, largely because people with high intelligence can learn the skills needed to be successful on the job more rapidly. However, due to legal concerns they’re not recommended for companies hiring in the US.” This is what a classic FAANG interview is: an IQ test which is disguised as a relevant skills test for legal reaso…

I'd rather have a hard worker with an average IQ than someone who's lazy with a high IQ.

Besides, IQ tests have biases that don't translate to good job performance. For example, if English isn't your first language you may do worse on an English-based IQ test.

Re: Making software engineering interviews predictive of job performance

#48

Post by thedance »

Havoc wrote: post #39

I don't think you can. Job performance is something that gets figured out on the fly. Unless you're working in a crazy standardized operation thats a moving target. e.g. We've got a ton of manager & they all sorta do their own thing...they gravitate towards their strong suit essentially. You can't really test for that in advance. Some people excel at leading 30 man teams on stable jobs. Others excel at being parachut…

This whole article is predicated on the idea that you can quantitatively measure the performance of a software engineer. That seems completely unfounded.

Re: Making software engineering interviews predictive of job performance

#49

Post by thedance »

tschiller wrote: post #46
Havoc wrote: post #39

I don't think you can. Job performance is something that gets figured out on the fly. Unless you're working in a crazy standardized operation thats a moving target. e.g. We've got a ton of manager & they all sorta do their own thing...they gravitate towards their strong suit essentially. You can't really test for that in advance. Some people excel at leading 30 man teams on stable jobs. Others excel at being parachut…

If you know what kind of company/situation you're in, you can certainly select for candidates that are more likely to thrive in that environment. As you mentioned, people gravitate toward their strong suits. So by asking behavioral questions, you can learn how they approached situations in the past and whether their strengths/weaknesses are the right mix for your needs.

If your company knows what situation it is in, and if that situation persists for more than a year, your company is already better than many. The company I just left had an endless parade of new missions, new "values", new career ladder descriptions, and the median tenure of managers in engineering was about 6 months. The idea that you can either rate the performance of your staff or predictively hire engineers who will succeed under these conditions is a joke.

Re: Making software engineering interviews predictive of job performance

#50

Post by bmn__ »

bitwize wrote: post #14
amznthrowaway5 wrote: post #11

Earlier quoted context omitted.

Why not just give an actual IQ test?

That opens you up to liability for race-based discrimination. The accepted wisdom is that IQ tests are inherently racist and favor white evaluees over black and brown ones.

That's formulated the wrong way. An IQ test per se cannot be racist because it is not a human being with intentional behaviour. This means blame must fall on the test designer or test conductor to introduce a racist biasing. In places where accurate results matter, the best available methods of psychometrics are used: http://enwp.org/Progressive_Matrices These are free of bias: culture, language, reading/writing ability does not matter. If these are used and assuming the test conductor did not screw up, and if white evaluees score better than black and brown ones, then it is an objective measurement of reality, not racism. Thus I think the "accepted wisdom" is wrong. If you know of any examples of racism through IQ tests, indicating intention, I'd be glad to hear of them.
Post reply on HN