Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has there been any academic research into software interviewing?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 129 posts

Re: Ask HN: Has there been any academic research into software interviewing?

#41

Post by mancerayder »

emaginniss wrote: post #26
lemony_fresh wrote: post #9

When I was self-studying social psychology I came across social facilitation theory and found it applicable to understanding interviewing. People have a hard time performing complex tasks while other people are watching because their working memory is monitoring the social situation. This varies for each individual. For tasks that have been rehearsed, like a musical or dance routine, they perform better. Politeness t…

I disagree completely about your theory. I have been on both ends of the whiteboard interview an almost uncountable time during my career. What it has always come down to in my mind is a collaborative process to solve a problem where the one performing the task is trying to solve the problem, while the giver of the task makes suggestions and points out weaknesses before they get out of control. I'm sure there are sad…

I've had whiteboard interviews where the interviewer stays silent while taking notes.

I failed 100% of them with various, strangely random feedback by HR/recruiter.

I've come to conclude it's the hiring mechanism of social incompetents / unempathetic sciency types. And it was always a dev from another group than the one I was applying for.

No thanks.

Re: Ask HN: Has there been any academic research into software interviewing?

#42

Post by dcolkitt »

barry-cotter wrote: post #32

I doubt there’s been too much on software interviewing specifically but interviewing/candidate selection for jobs is a large part of personnel psychology. A recent meta-analysis of what they know is below. Basically general mental ability (g, IQ) is the single best predictor, work sample tests work well, structured interviews can increase the performance of IQ tests or work samples and unstructured interviews are a t…

Can't agree with this enough. The most consistent finding in industrial psychology is that general cognitive ability is by far the best predictor of job performance in virtually every role ever tested.

Even jobs that don't seem cognitively demanding, like janitors or infantry. Higher IQ candidates almost universally do better, even if they start with much less experience.

The takeaway as it applies to modern software hiring, is that skillset is way overemphasized. It's quite common to see job ads that heavily focus on the company's specific tech stack. ("Must have experience with Rails, React, Travis, and AWS")

It's much better to cast a much wider net, and try to find the smartest people anywhere. High IQ people can easily re-tool their specific skillset. What's interesting is this is much closer to how the most successful tech firms, like Google, tend to hire.

Re: Ask HN: Has there been any academic research into software interviewing?

#43

Post by ghaff »

Scoundreller wrote: post #38
ghaff wrote: post #19

Earlier quoted context omitted.

I haven't looked for any recent research in this area but when I saw some way back when it was, in one sense, somewhat depressing. I forget if it was for undergrad or MBA programs but, as I recall, the correlations were basically to SATs/GMATs and class rank/grades. Of course, your results will vary depending on how you define success but basically quantitative hard measures had the most predictive value.

But what outcome were they measuring? Grades and rank in the MBA program? A good student will probably continue to be a good student. But how to select which students will become good practitioners may be a different story.

This was a long time ago and I don't remember the details. One commonly-used quantitative workplace success metric is compensation/salary, which is not necessarily unreasonable for MBA programs but can be more problematic in other areas.

And you're right. It's not really surprising that getting good grades/test scores at one level of school correlates reasonably well with the same thing at another level. Which I imagine is one reason universities/grad programs generally don't optimize purely on that metric because their objective isn't solely about cranking out students who study well in school.

ADDED: This, by the way, is more or less just a variant of general intelligence measures being correlated to a lot of outcomes. The SAT and so forth aren't intelligence tests, but I'm sure they're very correlated.

Re: Ask HN: Has there been any academic research into software interviewing?

#46

Post by lemony_fresh »

emaginniss wrote: post #26
lemony_fresh wrote: post #9

When I was self-studying social psychology I came across social facilitation theory and found it applicable to understanding interviewing. People have a hard time performing complex tasks while other people are watching because their working memory is monitoring the social situation. This varies for each individual. For tasks that have been rehearsed, like a musical or dance routine, they perform better. Politeness t…

I disagree completely about your theory. I have been on both ends of the whiteboard interview an almost uncountable time during my career. What it has always come down to in my mind is a collaborative process to solve a problem where the one performing the task is trying to solve the problem, while the giver of the task makes suggestions and points out weaknesses before they get out of control. I'm sure there are sad…

Would you allow junior developers to whiteboard a highly qualified and accomplished CTO candidate? No. It is acceptable for peers who will be competing with the candidate in the promotions tournament or the hiring managers to conduct whiteboarding exercises.

Re: Ask HN: Has there been any academic research into software interviewing?

#47

Post by throwayEngineer »

neilk wrote: post #5

Interviewing.io is a startup that does some research into candidate evaluation. To their credit, they have even posted research with negative results. http://blog.interviewing.io/posts/

I think reading 10-20 ancedotes is far more useful than an academic paper.

After meeting my PhD professor. I stopped trusting academia for truth.

Re: Ask HN: Has there been any academic research into software interviewing?

#48

Post by kayoone »

neilk wrote: post #5

Interviewing.io is a startup that does some research into candidate evaluation. To their credit, they have even posted research with negative results. http://blog.interviewing.io/posts/

I had the opportunity to be in a small interviewing workshop with Aline, she really knows what shes doing and I believe the tech interviewing world would be a better place if more companies adjusted their process based on Alines/Interviewing.ios insights.

Re: Ask HN: Has there been any academic research into software interviewing?

#49

Post by crispyambulance »

"job success" is something that happens in the future relative to a "job interview".

To be able to truly predict whether or not someone will be successful means that the interviewer would have to able to foresee how that person will interact with others in their workgroup , rise to challenges that don't yet exist, and be motivated to remain for a "long-enough" engagement with the company (whatever that means). Predicting the future is just damn hard, and it's even harder when nebulous desired outcomes such as "job success" are used.

To make matters even more difficult, this problem also has "another half" to it which people tend to ignore: the employer doing the interviewing.

These types of questions seem to take the point of view that an employer is presented with a bowl of fruit (candidates) and all they have to do is be able to select the best fruits.

But it doesn't work that way. You may not get the fruit which you want, that fruit may want to go elsewhere or others might have already grabbed the fruit you would have wanted. The fruit which looks good now may end up rotten a short time later. Some fruit that looks undesirable today, may be awesome later. You may grow tired of apples and desire pears, but you've already filled your pantry with apples.

What would happen if some organization was able to "figure this out" and truly maximally optimize their candidate selection using interview techniques? I am not so sure it would be easily distinguishable from what other similar competitive employers are doing. Predicting the future can only go so far.

Re: Ask HN: Has there been any academic research into software interviewing?

#50

Post by dudul »

One thing I've been wondering for a long time when I see the quasi-sado-masochist relationship between engineers and hiring processes is: are bad hires that 1) costly, 2) frequent ?

In my 12+ years career, I've worked with so-so engineers, but never truly bad ones that would ruin a project. And even the ones that weren't great, what damages did they really cause? I've seen many more companies failing because of a bad product, bad sales strategies, bad market fit, bad business model than engineering teams using the wrong programming language, or not building microservices the right way.

I find this constant obsession around identifying "rockstars", avoiding "bad" engineers at all cost to be truly unhealthy.

Post reply on HN