Coding interviews are stupid (ish)

Perhaps they meant to say "studying for a few weeks to solve fun puzzles, after getting a CS degree from Stanford, moving to a city where a basic apartment costs $1 million, and getting 8-10 years FAANG experience, also like half that $400k is stock not cash"

But no licensing!

I find a simple realignment would solve a lot of this interview angst: Interviewers seem to interview on the premise that they need to find out what the candidate can't do. But this is not useful. I can tell you what they can't do. To a first approximation, the answer is "everything". I am an experienced and skillful developer, yadda yadda yadda, and I've never touched React, have no game development experience, have never written kernel code, haven't touched matrices since school, have only dabbled in embedded development as a hobbiest, etc. etc. etc. Make a list of all the things I can't do and I look like an idiot. The list of possible things is too long.

The goal of an interview is to find out what the candidate can do.

If you interview someone, and they "fail all the questions we asked", it is not the candidate who has failed... the interviewer has failed. You ran an entire interview and all you know is what the candidate can't do. You have learned virtually nothing. The questions must be adjusted to what the candidate can do. Only for the absolute worst candidates should you ever exit an interview saying they failed at everything. (Sadly, such candidates do exist. For instance, consider the recent stories about AI fraud. If your level of programming skill is "I have ChatGPT", I'm going to have a hard time scoping a question down to you.)

If I had asked this question in an interview, or a related one, I would have stepped the problem down until the candidate could pick it up. (Odds are I'd have started lower and worked my way up anyhow.) If the candidate then sort of finds their footing and once going can start folding in the other requirements as we go, great. Who sits down and designs a system for all twelve adjectives ("reliable", "concurrent", "redundant", "runs on a Z80") we want anyhow? One can not help but design a system one adjective at a time, with the others only kept in mind to make sure we don't back into a corner. There's no reason not to interview that way too. (And I tell the candidate this is what we're going to do, so hopefully they don't feel like I'm out to get them or experience requirements whiplash without knowing why I'm doing this.)

I’ve been having trouble finding the scientific papers behind this theory, but one I’ve latched onto is that “filtering against known strong signals is helpful” but also that “additional filtering beyond that is more harmful than random choice of the remaining pool”.

Basically, it can probably be shown that if you hire a group of completely random sample of resumes vs. hiring a random sample of people who can produce a working fizz-buzz program, that the candidates from the latter group will perform better. But if you then filter the fizz-buzz group by “can they solve this negative base math problem?” you’ll be removing a disproportionate amount of candidates who would have turned out to be excellent for your organization, and your final “super-candidate” pool will actually be weaker (for your org / that position) than the average of all those who could solve fizz-buzz.

I think the research shows that you should filter based on what you know for sure provides a true signal, then select randomly from the pool which passed your known filters. If anyone can help me find anything relevant about this, I’d appreciate it.

Too many orgs treat hiring like the “secretary problem” but that requires the assumption that you can grade everyone accurately on a continuous scale. We can’t yet do that with software engineers - theres no way to say someone is “80% awesome” vs. “93% awesome”.

Yes this, and to further this, the "can't do" thing that you measured is in fact extremely partial and incomplete: you've measured that the developer can't write an algorithm on a whiteboard with a felt tip marker in front of a stranger they just met. You don't even know if they can't write the algorithm. Just that they can't do it there.

Is that important to you? Maybe it is for some people. As a person who has been in team lead and hiring capacity before, it is not for me.

I trained for interviewing at Google twice, but never really chose to give interviews despite that being a highly pressured part of the job because I could not philosophically vibe with that process. But what some people who have adopted this process are missing is that Google etc does this only because they are swamped with resumes and have their pick of bazillions of quality engineers. They don't care about false negatives, more false positives.

Startups and smaller companies should care about false negatives. It's hard to find and retain good people. Smaller companies need to aggressively find and cultivate good people to make good teams in order to be/stay competitive -- and that means accepting a diversity of ways of working.

> - licensing ensures only a minimum level of quality… people with licenses still do interviews, often just as grueling

Agreed, I never asserted against these two points. The point of licensure is to make the first round, which is fairly routine at this point, more equitable and less susceptible to probabilistic effects. It also frees labor from administering this exam round to every candidate.