Live data from Hacker News

Making software engineering interviews predictive of job performance

qualified.io

21–30 of 113 posts

Re: Making software engineering interviews predictive of job performance

#21
post #4

In the end, it's a crap shoot... Even their highest marker of validity is only about 50%. In the end, if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel. Market pay will also vary wildly and dramatically, and perception is equally varied. Where I work now, we have a small pre-interview code challenge..…

> if you make it too hard in an employee market, you will have good to great candidates drop off... if you make it too easy, you will wind up at the bottom of the barrel And where you set the bar will depend on, roughly speaking, your prestige. Google can afford to be extremely selective when hiring technologists. Similar thing goes for Harvard and the Air Force. > what a person works on (developers) outside of educa…

All teams should be selective. But if you don't have a Google-sized brand and budget, you need to develop a clearer understanding of your teams strengths/weaknesses and needs.

This understanding will enable you to figure out which hiring criteria do you need to maximize vs. which do you need to satisfy and where the bar is.

Also, by interviewing more effectively, you punch above your weight by spending more face-to-face time selling your company/team/role.

Re: Making software engineering interviews predictive of job performance

#22
"Hard Interviews, Happy Workers". This note near the end assumes correlation shows causation, which it does not. You could also argue that more workers want to work in certain jobs, creating more competition and leading the company to make interviews harder. The workers in those jobs may be happier, but not because the interviews were harder.

Re: Making software engineering interviews predictive of job performance

#23
Some of the points here are good, but I think some of them are too simplistic.

This article makes a huge assumption: that performance is the only factor you're hiring for, and therefore unstructured interviews are worthless. That would make sense if you're interviewing for a factory of emotionless robots. But humans are social beings, and human performance as a team is more complicated than a sum of individual productivity. You need to take into account the affect that the hire would have on the entire team. Just about everyone has an anecdote about a coworker sometime during their career that was super productive, but completely demoralizing to the team. This is a very valid reason to give unstructured interviews, and if you ask smart people who give them, this is what they're looking for.

I totally agree with the utility of work-sample tests. There is nothing better to see someone's performance than simply giving them a small but relevant chunk of work to do. It is very important to keep them short and flexible though.

If you make them too lengthy, you'll skew the results against people who have better things to do with their time. What kind of person works for free anyway? Certainly not the caliber of people I want to work with.

And if you make the test too specific to certain technologies, you skew the test against smart engineers who are less familiar with that tech. If you want a problem solver, give them a problem in plain words, and let their solution be open-ended.

I don't agree with making tests particularly hard, though. Make it nuanced, yes, but of normal difficulty for a given assignment. You need the appropriate people to do the job. If you only hire top-tier algorithmic experts for your basic CRUD app, then you're going to have a very expensive CRUD app with a bored and demoralized team. If someone does sloppy work, you can spot this on a normal exercise as much as you can on an unreasonably difficult one.

Re: Making software engineering interviews predictive of job performance

#24
I think the important thing is to realize that you need to rely on realistic factors and many different signals to get useful data.

I've been to plenty of interviews for "senior software engineers," where they used the standard gauntlet of algorithms/problem solving/data structures questions in rapid succession for eight hours.

I tend to deliver most of my projects on time and of a high quality. I value things such as correctness and maintainability, and am conscious of other factors to consider such as viability and availability of time and capital. I spend a good amount of my time writing and seeking feedback from colleagues and think it's important to share our ideas and talk about our work.

And when I practice my work I don't think I've ever been in a time-constrained scenario where I had to come up with an implementation of an algorithm while being judged and evaluated for how I think. I know enough about my work to understand when I need to use BFS or DFS and when to use a tree, a map, or a heap; but I also know that if I can't think of it off the top of my head I can walk over to my library and pull out a book or look it up online.

I'm also a proponent of formal methods. I've written specifications to help solve race conditions and validate implementations of locking algorithms. I can write basic proofs.

And I'm a great writer as a result. I spend a good amount of my time writing and communicating my ideas, seeking feedback from colleagues, and in general improving my understanding of problems.

How does finding a cycle in a sorted array or reversing a linked list signal to you whether I would be a good fit for your company as a senior engineer? Does your company also value clear writing and good communication skills? Do you value engineers that can manage themselves and get work done autonomously?

Modern interview practices at most places I've been through don't know how to get those signals. It's useful knowing that the people I work with are also familiar with when to use a heap versus an array but I also value other things at the senior level that most companies aren't really testing for.

I always prefer data over gut instinct but I feel like the process of hiring people is not amenable to clean signals... it's a very messy process and I'm not hopeful that there will ever be a silver bullet.

Re: Making software engineering interviews predictive of job performance

#25

I think the important thing is to realize that you need to rely on realistic factors and many different signals to get useful data. I've been to plenty of interviews for "senior software engineers," where they used the standard gauntlet of algorithms/problem solving/data structures questions in rapid succession for eight hours. I tend to deliver most of my projects on time and of a high quality. I value things such a…

What does your interview process look like?

Re: Making software engineering interviews predictive of job performance

#27
post #14

Earlier quoted context omitted.

That opens you up to liability for race-based discrimination. The accepted wisdom is that IQ tests are inherently racist and favor white evaluees over black and brown ones.

Wouldn't coding interviews show the same bias?

The difference is, the Supreme Court has already made a ruling on IQ tests.

Re: Making software engineering interviews predictive of job performance

#28

“General Mental Ability (GMA) tests (like the IQ test) are very predictive of future job performance, largely because people with high intelligence can learn the skills needed to be successful on the job more rapidly. However, due to legal concerns they’re not recommended for companies hiring in the US.” This is what a classic FAANG interview is: an IQ test which is disguised as a relevant skills test for legal reaso…

by "classic" do you mean the "if you were stuck in a blender?" questions? you haven't interviewed in a while if you think they're asking those. it's all standard DS&A whiteboarding, etc. there's nothing even close to resembling a general aptitude test

Re: Making software engineering interviews predictive of job performance

#29

“General Mental Ability (GMA) tests (like the IQ test) are very predictive of future job performance, largely because people with high intelligence can learn the skills needed to be successful on the job more rapidly. However, due to legal concerns they’re not recommended for companies hiring in the US.” This is what a classic FAANG interview is: an IQ test which is disguised as a relevant skills test for legal reaso…

FAANG coding interviews are definitely not IQ tests.

IQ tests (for interviewing) are typically spatial comparisons, some math, logic puzzles, reading comprehension, and pattern recognition and are done so under a time budget of around 30s to 1 minute per question

FAANG coding requires CS basics - hashmaps, arrays, sorting strategies, trees, recusion, basic ops for all of the data structures, and a few other concepts. There are typically only a handful of questions with 60 - 90 minutes per question

Re: Making software engineering interviews predictive of job performance

#30

so you're gonna pay us to do these "work samples" at market rate, right?

Sarcasm aside, this is exactly why we make ours as short as possible in duration. You don't want to filter out people with valuable time.
Post reply on HN