Lessons from 3,000 technical interviews

To get a job at a company you have to be given the opportunity to interview, and to succeed at the interview. If you only evaluate performance at the job level, you don't evaluate performance of anybody who failed the interview. This is one of the fundamental challenges in hiring - you have a biased sample when trying to determine how effective the hiring process is at screening candidates.

Interesting bit on the MS degree. I followed the link, and I'm not quite as surprised that the correlation is poor, or even negative, given the way the data was collected and analyzed.

Absolutely agree that some MS degrees are pretty much less rigorous cash cows by now, that allow students to skip the fundamentals such as data structures, operating systems, and compilers.

However, many CS MS degrees actually do require this as a background, to the point where some programs have emerged to prepare non-CS majors for MS degrees, kind of like those post-bac premed programs. It's hard to believe that those MS degrees, which require a decent GPA in those core courses, along with high GRE scores (sorry, but we are talking about interviewing skill, which may be more related to exam taking ability than job performance), wouldn't result in a similar profile to people with CS degrees from top schools.

This is fully acknowledged in the text of the article referenced in a link, but unless people follow it, I do think the message may be a bit misleading.

That's an aside, though. The value may very well be in the prep for these degrees (ie., the post-bac CS coursework required for admissions to a reputable MS program). If you can get that through online courses (udacity or coursera) through genuinely rigorous self-study? Yeah, that might do it, for far less money. I've audited a few of them, and they're the real deal, that's the real coursework there.

Most of the things that make us better during our career have nothing to do with programming faster: If anything, they can make us program slower, and do worse in interviews.

For instance, an interview I took last year came with a premise that I found downright bonkers. It was based on code the company had in production, but it stopped being a good problem to look at years before. I was having trouble coming up with the right tradeoffs for the implementation because all my experience was telling me that the entire approach was misguided in the first place, so the problem should not be solved. I passed, but it was a far rougher performance than I would have liked.

There's also how being far from college makes the least important knowledge gained fade away, and the least important thing I studied was memorizing algorithms. I write new algorithms at work sometimes, and I implement off the shelf ones too, but I don't have to recall them off the top of my head. Nobody has to implement distributed consensus algorithms under time pressure, or write HyperLogLog from memory.

So ultimately, there's what easy to measure, and then there's what is valuable and important. We go with what is easy, and those are things that are taught in college. Understanding the right level of testing or designing a system for observability are far more valuable in the long run: It's crazy how much downtime in well known companies comes from people not learning those things in college. But since we are bad at measuring those things, and kids right out of school don't know them, we don't interview for that.

And sadly this is why we all end up hiring by network so much: We can't tell if someone is good in a day, and we can't really ask people to dedicate two weeks to work with us in a probationary period if they have real jobs, but we sure can recall quality former coworkers and ask them to join in.

For two reasons:

1. From the perspective of a job seeker, the interview is what gets you the job, so it's in your interests as a job seeker to learn how to look better as a candidate based on data about how interviewers judge candidates.

2. The purpose of an interview is to be a predictor for job performance, so while this article doesn't address this part of the question, you want interviews to predict job performance, and how to make interviews do that is interesting.

It's not just 'impact' it's 'correlation' (or both).

Universities don't teach programming, so it's not surprising you don't get applied skills out of the box with a degree.

'Good courses' are focused directly on teaching specific programming skills.

I think they are a good idea because learning programming by 'osmosis' (as you go along) can result in 'not knowing' a lot of key things, even when they are right in front of your face the whole time.

Most importantly - people who are going to take the time to specifically learn new skills, are displaying the kind of conscientiousness that you want in your talent - learning the actual skills is another benefit of that.