Nice analysis.
In my hiring process, we use a number of filters to gather the data we're looking for to make a decision. That requires a bunch of different steps. By the time we're done, we've spent at least 8-10 hours talking with this person.
From a technical perspective, we do the following:
- A short screen to go over the resume and ensure we've got a rough fit to the right role.
- A simple consistent coding exercise using coderpad. (surprisingly some fail)
- A series of consistent open-ended questions we ask everyone about their tech background, such as, what was one of the most difficult bugs you ever fixed?
- A set of consistent design/architecture problems: "given this design, what problems do you see? How would you fix them?"
- Another consistent, more involved coding exercise with an existing code base that is VERY much related to the work they'll be doing.
- A Q&A session on a wide range of technical topics. Goal is NOT for someone to know everything, it's to get a bit of map of their strengths and weaknesses. We found we make assumptions of what someone knows based on our background and their resume. We try to make this fun.
- Another set of behavioral and situational questions with a shared scoring rubric on values such as teamwork, collaboration, communication and leadership.
And then we have to take all of that data and look at it holistically and across a wide range of candidates. And even then we'll make mistakes, but we keep trying to optimize it, reduce bias, and make it better for candidates and us alike.
One last note: I like to finish my first interview with the question:
"Is there anything about yourself that you really want me to know that we haven’t discussed?"
Because I know I've only had ~40 minutes to get to know this person. I have my agenda of what I want to know, but I could easily miss a lot. So I want to give them a chance to represent themselves in the broadest way possible.