I have learned that an (in)ability to program "in the small" correlates very well with an (in)ability to program in the large, and now ask mostly simply questions whose answers are things like one-line Boolean predicates to test for well-defined conditions. It is paradoxically easier for an inept candidate to fake his way through an algorithm design question than it is to fake the coding of a simple test for "determine whether two closed intervals [a,b] and [x,y] overlap each other".
Technical interview performance is kind of arbitrary
31–40 of 246 posts
Re: Technical interview performance is kind of arbitrary
#32Great post as always Aline. I'd be most curious as to how well anonymity was kept. Did interviewees identify their employers, schools, or any other information that might create bias while in the interview itself? I've been recruiting for a long time, and I'm rarely shocked about the result of an interview - maybe a few times a year. There are tons of possible explanations for that, and lots of possible explanations…
Most of our interviews didn't have people unmasking until after the feedback step.
Re: Technical interview performance is kind of arbitrary
#33Earlier quoted context omitted.
most other job interview processes don't involve solving a series of riddles
In most other jobs, candidates don't vary in skill by a factor of 10x or more
Lawyers were mentioned directly in your grandparent comment. Sales and athletics also easily show at least that much variation from person to person. Business owners show much, much more variation than that, even if you restrict the field to very similar businesses. Researchers show variation similar to business owners (i.e. mind-bogglingly high).
You can easily construct differences of larger magnitude for what you might think of as "simple" jobs like telephone receptionist; a fluent english speaker is going to be more than 10x as productive in that job, assuming they're supposed to do business in english, than someone who can barely get by, and probably billions of times more productive than someone who can't speak english at all. There are a lot of Chinese people out there whose highest ambition is to "work in an office". I've met some of them! Most will never achieve that goal, because they don't have the requisite skills.
Most people who don't speak english won't apply for a job answering a telephone in english (but a lot of them will! This is exactly the kind of customer service representative everyone has grown to hate). I'd like to see some software job openings that specified a skill set, and were willing to hire people with that skill set. That doesn't seem to be the direction people are headed in, though...
Re: Technical interview performance is kind of arbitrary
#34With programmers, the single easiest way to identify good candidates (in my experience) is sheer interest in what they do / desire to learn. This is a learn everyday field and if you're interested in what you're doing, you're going to do a lot better at it. It's hard to apply yourself mentally to something that you don't have a good level of interest in. Given that it's a learn everyday field, people with that level…
Interest is not enough, unfortunately. There are plenty of engineers who are attracted to the challenge and excitement of building new things, but have no appreciation for The Right Way to build things. Great engineers think beyond "how" and ask the "should" questions as well. Mediocre engineers glue things together in a haphazard way with little thought about what's the best way to write things. Caring about maintai…
Attention to detail will usually preclude someone from gluing things together like you describe though. Those are important details to account for and the type of person you're describing is more task focussed (aka - I got it done) than detail focussed (aka - I got it done right).
Re: Technical interview performance is kind of arbitrary
#35Take home tests FTW. The thinking goes as follows: 1. If the candidate can't be bothered to complete a 2-4 hour (depending on claimed seniority) code test in the language of their choice, we can't be bothered to talk to them. 2. If the candidate does reasonably well by completing the code test somewhat on time (with a fat margin allowed for them, well, having a life) and within parameters of the task, they're invited…
You need to verify that the candidate actually did the take home test, and didn't hand it to their more talented friend.
Re: Technical interview performance is kind of arbitrary
#36In the HN echo chamber, there isn't a day without some blog post/article describing how our interview process is BS, interview is broken, etc. I don't necessarily dispute this state of affaire, but does anyone know how it compares to other fiels/professions? How about interviewing a lawyer? Or a doctor? Or an account manager? Or a product marketer? Are developers the only one with a "broken" interview process?
You could argue that building software has a higher potential for complexity than work in any of those fields. So how do you measure someones ability to be good at that within a reasonable time frame using limited resources (interviewer time)? You're always going to have a "broken" process with those limitations, because you can't test for every possible scenario they may encounter. This leads to a small subset of questions that you use to generalize ability, which leads to errors.
Re: Technical interview performance is kind of arbitrary
#37Most interviewers don't ask enough technical questions to have any idea what a candidate knows or doesn't know. If their one or two questions happen to be something the candidate knows well, they'll call them a genius. If they happen to not know, they'll label them an idiot. You can learn a lot more from 20+ rapid fire questions than forcing a candidate to eek out an answer to something they're not familiar with. And…
Along these lines, I start out with very broad questions. Something like, "tell me how you'd troubleshoot a web service that's suddenly not accepting connections / suddenly performing badly." Different candidates will focus on different aspects of that problem depending on their background: low-level networking, cloud environments, application-level problems, databases etc. Based on their resume, I like to see if the…
I have a similar question to this that I like to use - "Tell me how software gets delivered where you work now (or a place you've seen it done well). Idea to Customer."
Where the candidate starts, stops and goes into detail are usually quite interesting.
Re: Technical interview performance is kind of arbitrary
#38Earlier quoted context omitted.
Along these lines, I start out with very broad questions. Something like, "tell me how you'd troubleshoot a web service that's suddenly not accepting connections / suddenly performing badly." Different candidates will focus on different aspects of that problem depending on their background: low-level networking, cloud environments, application-level problems, databases etc. Based on their resume, I like to see if the…
This is a very good way for senior engineers to interview candidates. Newer engineers interviewing candidates will typically have difficulty doing this well.
That's fair - I wouldn't be a good judge of this if I didn't have really good breadth myself. Often a candidate in his own are of expertise will go beyond what I can fairly judge and I end up having to do a bit of reading afterwards to confirm and / or assume they were correct.
Re: Technical interview performance is kind of arbitrary
#39Take home tests FTW. The thinking goes as follows: 1. If the candidate can't be bothered to complete a 2-4 hour (depending on claimed seniority) code test in the language of their choice, we can't be bothered to talk to them. 2. If the candidate does reasonably well by completing the code test somewhat on time (with a fat margin allowed for them, well, having a life) and within parameters of the task, they're invited…
I assume the take home test is after you have talked to them? I wouldn't spend multiple hours unless it was already clear there was some desire to move further
Re: Technical interview performance is kind of arbitrary
#40Take home tests FTW. The thinking goes as follows: 1. If the candidate can't be bothered to complete a 2-4 hour (depending on claimed seniority) code test in the language of their choice, we can't be bothered to talk to them. 2. If the candidate does reasonably well by completing the code test somewhat on time (with a fat margin allowed for them, well, having a life) and within parameters of the task, they're invited…
You need to verify that the candidate actually did the take home test, and didn't hand it to their more talented friend.
That's why you get them to talk you through their process during the face-to-face interview. It'll be immediately apparent if they didn't do the work themselves.