Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has there been any academic research into software interviewing?

news.ycombinator.com

Post reply on HN
Print view View original

81–90 of 129 posts

Re: Ask HN: Has there been any academic research into software interviewing?

#81

Post by dcolkitt »

kayoone wrote: post #53
dcolkitt wrote: post #42

Earlier quoted context omitted.

Can't agree with this enough. The most consistent finding in industrial psychology is that general cognitive ability is by far the best predictor of job performance in virtually every role ever tested. Even jobs that don't seem cognitively demanding, like janitors or infantry. Higher IQ candidates almost universally do better, even if they start with much less experience. The takeaway as it applies to modern software…

I can't agree with this. High IQ can't be everything. Wouldn't this value experience and knowledge (like everything you learned in university) at zero? Imo there are a lot of great/amazing and productive Software Engineers that wouldn't do well in a whiteboard algorithm interview. The people that tend to do well, are the ones that have a lot of practice with the kind of problems.

So, what we know from industrial psychology isn't that experience doesn't add value. It's that high IQ people learn faster. This makes sense when you remember that intelligence is broadly defined as the ability to learn.

While Alice may have fewer years of experience than Bob, she might have effectively more experience if she absorbed understanding at a faster rate. An unexperienced, high-intelligence person usually starts at an initial disadvantage, but manages to "get up to speed" quickly.

This also underscores the particular importance of intelligence in software. The field is constantly awash in new technologies, where nobody has had time to accumulate extensive chronological experience. So, it's really important to find people that can absorb new concepts quickly.

[1] https://www.sciencedirect.com/science/article/pii/S019130851...

Re: Ask HN: Has there been any academic research into software interviewing?

#82

Post by lmkg »

barry-cotter wrote: post #32

I doubt there’s been too much on software interviewing specifically but interviewing/candidate selection for jobs is a large part of personnel psychology. A recent meta-analysis of what they know is below. Basically general mental ability (g, IQ) is the single best predictor, work sample tests work well, structured interviews can increase the performance of IQ tests or work samples and unstructured interviews are a t…

DO NOTE that the in the United States, the use of IQ tests for hiring is potentially legally fraught.

A SCOTUS ruling[1] has found that IQ tests are assumed to disfavor minority employees, and therefore using them as a major factor in hiring decisions may run afoul of the Civil Rights Act. There are ways around this, but generally speaking you have to prove the specific test you are administering either does not disfavor minorities, or else show that it is directly related to the specific position you're hiring for.

[1] https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co.

I am not a lawyer, do not take legal advice from randos on the interwebs. You should consult with a Real Actual Professional on how to deal with this risk. There have been both laws and court cases since the above that impact the ruling, which I am not qualified to analyze.

Re: Ask HN: Has there been any academic research into software interviewing?

#83

Post by jonfw »

klntsky wrote: post #76
pps43 wrote: post #65

Earlier quoted context omitted.

> Wouldn't this value experience and knowledge (like everything you learned in university) at zero? Having switched professions twice, I now use practically nothing of what I learned in university from about second year forward. Math and to some extent physics are still relevant, but that's pretty much it. It might be painful to admit, but the practical value of that rather specialized knowledge that took me several…

> It might be painful to admit, but the practical value of that rather specialized knowledge that took me several years of hard work to obtain is pretty much zero now. It's sad that your professors didn't tell you beforehand that you were going to learn how to learn during that time, not just study a particular technology stack.

This bears the question- are classes based around studying a particular technology track the most effective way to teach somebody how to learn?

Re: Ask HN: Has there been any academic research into software interviewing?

#84

Post by wodenokoto »

neilk wrote: post #5

Interviewing.io is a startup that does some research into candidate evaluation. To their credit, they have even posted research with negative results. http://blog.interviewing.io/posts/

Has anyone here done interviews with interviewing.io? How was it, did it lead to a job?

Re: Ask HN: Has there been any academic research into software interviewing?

#85

Post by ghaff »

jonfw wrote: post #83
klntsky wrote: post #76

Earlier quoted context omitted.

> It might be painful to admit, but the practical value of that rather specialized knowledge that took me several years of hard work to obtain is pretty much zero now. It's sad that your professors didn't tell you beforehand that you were going to learn how to learn during that time, not just study a particular technology stack.

This bears the question- are classes based around studying a particular technology track the most effective way to teach somebody how to learn?

The short answer is probably no. And the fact is that top universities mostly aren't so focused on teaching whatever language or framework is the flavor of the day.

That said, you need tools of some sort if you're going to actually build things as opposed to just learning, say, algorithms in pseudo-code. And it probably makes sense to use some fairly standard language to do so. There's not much point in making things deliberately obscure by making students use some language that the professor designed for his PhD thesis.

Re: Ask HN: Has there been any academic research into software interviewing?

#86

Post by mratzloff »

dudul wrote: post #50

One thing I've been wondering for a long time when I see the quasi-sado-masochist relationship between engineers and hiring processes is: are bad hires that 1) costly, 2) frequent ? In my 12+ years career, I've worked with so-so engineers, but never truly bad ones that would ruin a project. And even the ones that weren't great, what damages did they really cause? I've seen many more companies failing because of a bad…

I strongly agree. A good engineer can compensate for bad product management and occasionally bad leadership, but can't do much about all the other things you listed.

There's no way to guarantee a successful engineering hire, but organizations try to because their culture is so blameful or fearful of conflict that all the incentives are misaligned.

I believe most organizations copy the Google, Facebook, etc. hiring processes with the goal of adopting an "industry standard process" because they don't know how to conduct an interview process that's tailored to their own organization, and they don't want to think too deeply about it.

Having a coherent interview process means dealing with your organizational baggage: understanding it and either resolving it or crafting the interview process to select for people who can integrate well with it. Usually it's the latter, since most of the issues start at the top.

Google, Facebook, etc. have spent a lot of time and money crafting a good interview process for their organizations: specifically what they value and don't value (even if the individual interviewers aren't always fully aware of what the organization is selecting for). They can also support a lot of false negatives.

But the copycats don't have those same needs or candidate pool, so it always comes across as a bit disconnected to me.

Re: Ask HN: Has there been any academic research into software interviewing?

#88

Post by gwbas1c »

lemony_fresh wrote: post #9

When I was self-studying social psychology I came across social facilitation theory and found it applicable to understanding interviewing. People have a hard time performing complex tasks while other people are watching because their working memory is monitoring the social situation. This varies for each individual. For tasks that have been rehearsed, like a musical or dance routine, they perform better. Politeness t…

> "if you want to work here, you are going to do what I say."

I recently started pointing that out, in a roundabout way, when I run interviews.

Why? If a candidate starts arguing about xml versus JSON, the candidate misses the point when I bring up xml. (Hint, I only discuss xml because it allows discussing widely known concepts that are very general and have little to do with xml versus JSON.)

Or, if a candidate says to me, "why can't we use async-await" in a particular question, the candidate also misses the point of the question. (Hint, I'm trying to test knowledge of concepts that's easy to test when doing something without async=await.)

Interview questions are always contrived and end up as an "if you want to work here, you are going to do what I say" exercise. If a candidate can't work within the constraints of a short interview exercise, then what will happen when the constraints of the job come into play? We never get to use our favorite patterns and APIs all the time.

Re: Ask HN: Has there been any academic research into software interviewing?

#89

Post by pmiller2 »

leeny wrote: post #73
pmiller2 wrote: post #55

Earlier quoted context omitted.

I don’t really see what’s special about their process other than the resume-blindness aspect. That’s been validated elsewhere by Triplebyte. Other than that one aspect, the process is exactly like your standard technical phone screen, except that you have to do a really easy Hackerrank problem to get invited to the platform. If I am wrong here, someone please correct me.

Hey, Aline here. Unlike Triplebyte, we offer people free, anonymous mock interviews with engineers from companies like Google, Facebook, etc. Basically, you get on the platform, practice, and then if you do well in practice (again real interviews not coding challenges), you can book real interviews top companies. Those interviews are also anonymous, which means that if you do poorly, you don't have to unmask.

Well, I disagree with your use of the word “anonymous.” Gender, national origin, and ethnicity are frequently easy to guess (at least at the level of “is not a white, American-born male”) via voice. I know you tried voice masking and it didn’t help equalize the results wrt gender, and I don’t have a better suggestion, so I’m not blaming the process, for not attempting to remove bias. This criticism applies to both the mock interviews and the real interviews.

Bias creeps in in other ways, too. I have done both on interviewing.io as a candidate. You probably remember how Instacart has interviews on your platform under the terms that it was just an informational chat, and anyone who met the criteria to interview would then get a real technical interview. I did the informational chat and was then not interviewed after deanonymizing. And, BTW, after I emailed support I never got my technical interview, nor was I given any sort of resolution or information about what happened.

It’s a good experiment, but it falls short of what I’d call “anonymous” and doesn’t really remove very much bias in the overall process.

Re: Ask HN: Has there been any academic research into software interviewing?

#90

Post by hunter23 »

dcolkitt wrote: post #42
barry-cotter wrote: post #32

I doubt there’s been too much on software interviewing specifically but interviewing/candidate selection for jobs is a large part of personnel psychology. A recent meta-analysis of what they know is below. Basically general mental ability (g, IQ) is the single best predictor, work sample tests work well, structured interviews can increase the performance of IQ tests or work samples and unstructured interviews are a t…

Can't agree with this enough. The most consistent finding in industrial psychology is that general cognitive ability is by far the best predictor of job performance in virtually every role ever tested. Even jobs that don't seem cognitively demanding, like janitors or infantry. Higher IQ candidates almost universally do better, even if they start with much less experience. The takeaway as it applies to modern software…

What is the correlation between IQ tests and algorithm interviews? Do people with better IQ scores do better on algorithm interviews?
Post reply on HN