A Method I’ve Used to Eliminate Bad Tech Hires

That sounds like a great way to do it; if I were an applicant, I would have liked process. (I am currently looking for jobs, and do have a recent experience of an on the spot test which I just hated).

I thought I'd try your step #2: I have no idea of the wall follower rule, and no prior experience in maze solving, but did bring it home in 18 minutes.

That included figuring out this particular Blockly UI (have admittedly seen similar UI with Dash and Dot, only at a much simpler level of doing non-branching, sample tutorials with my 6 year old boy), trying to design a simple program, reaching some dead ends / observing effects, and then realising what needed to be fixed.

I did use all the blocks for 17 lines of JS though, so perhaps better ways to do it, but thanks for that interesting little challenge at any rate. :-)

I'm guessing really smart people would have it done in just a few minutes.. Also monitoring son's cough (night time in Australia) to make sure it didn't get worse, for a bit of distraction.

(Edit: Brought it down to 1 spare block / 16 lines having a 2nd look after writing this comment.)

The Blockly maze thing is interesting, but you should realize that you're implicitly biasing against people with poor spatial skills.

My ex-wife is a concert-level pianist who can remember texts (words) after only hearing them once. She's also, astonishingly bad at spatial tasks (gender stereotypes aside).

So unless you're hiring for computer graphics, or visualization or some-such, which has a clear spatial component, you're implicitly biasing against less-spatial people.

This is good for testing for skills but does not help with team fit and behavioral issues down the road. I hired a person a while ago who provided with a clean solution to the problem with good separation of concerns and unit testing. I hired this person but after few months this person starts yelling at other engineers, won't complete the tasks. Ultimately, I had to let this person go which was very unfortunate given the quality of code. The headline is don't miss out on testing for team fit and other cultural values of your team.

There's no chance I would consider a job that required me to do this.

What does it mean "to have knowledge"? How much information qualifies you to be a knowledgeable? Plenty of us have a knowledge. Until, for one of your project you have to use Spring Boot and Hadoop at the same time. You know them well, you work with them already. But then you put both libs into one project and something throws an exception somewhere deep down in the unknown code. This is unexpected and does not come from your code. Good luck with searching for solution by yourself. Not that it's impossible, and you would learn something, sure, but troubleshooting takes a lot of time, you'd have to keep your current tasks aside and probability is quite high that someone had that problem before you.

IMHO, what makes you a good software engineer is not an encyclopedic facts in your head but general knowledge about principles, algorithms, patterns, etc. You don't have to remember all methods of all classes of standard library, for details you can check the web or use IDE. But if someone asks you about thread safe objects you have to know what does it mean. What may go wrong if you fail to make it so? Why properties must be immutable? And what the hell immutability is in the first place? All other things you should be able find in the online documentation / language reference.

What makes you a good software engineer is the ability to ask a meaningful questions. We have a powerful tools at our hands these days. Why wouldn't we use them for our benefit? You don't have to remember everything. It is there, within a hand's reach. Sometimes I have a feeling that asking Google the right question to solve your problem should be another skill tested during interviews.

What makes you a good software engineer is the ability to switch the tools and adapt quickly to new environment. Give him Internet, give him StackOverflow and ask him to solve a problem in new language, new framework. If he got the concept well, if he knows principles, that should be more than enough to jump in. Don't get me wrong here. I not going to underestimate guys with years of experience. I am not saying he will write flawless code. But he will solve the problem - that's a good start, isn't it?