The Terrible Technical Interview • Hacker News Board

Re: The Terrible Technical Interview

#201

Post by tptacek » Sun, Mar 22, 2015, 4:22 PM UTC

politician wrote: post #135

jes5199 wrote: post #124

Earlier quoted context omitted.

personally, I think the SVN question is a pretty good example of "can you talk intelligently about any aspect of any complex system you've ever worked with?". If a candidate can explain one , I'll recommend to hire them. The resume provides a good list of possible subjects to discuss, otherwise I just cast about randomly, try to chase threads in the conversation until I find something with some depth.

I completely agree with this - asking about VCS is a cross-cutting question that nearly all candidates have experience with unlike, for example, graphics, desktop, or cloud experiences. So, requesting that the candidate explain their VCS workflow is a way to generate a practical and relevant data point to use when evaluating multiple candidates. For example, the 1-in-a-million candidate that has used git with a CI/CD…

I've carefully read every comment on this subthread, and here's what I think:

This question is a trifecta of ineffective candidate screening tactics:

(a) It's a technical screening question, one a strong candidate could get wrong, based on a technical aptitude that is trivial to teach on the job and thus rarely worth paying a premium for.

(b) It's a subjective technical question, for which reasonable engineers can have differing opinions, which means it's an outlet for interviewer subconscious bias. Did the interviewer just eat lunch? Candidates will do better on this question if they have.

(c) It's a tea-leaf-reading question about engineering/team management: it's superficially and overtly about technology, but subtextually about a bunch of other things. Let's hope the candidate realizes that.

A typical interview lasts about 60 minutes. Let's say it takes 10 minutes to pursue this particular line of inquiry. That's 16% of your interview you're spending with a question that greatly rewards people who are good at talking about technology. Worse, if you ask that question early to a quiet but excellent candidate, you can psych them out, which means you pay for that version control question in every other question you ask.

It is totally reasonable to assess soft skills and team compatibility. But you have to design an interview that does it. You can't improv it based on candidate resumes.

Avoid questions like this.

This thread started with someone saying they're pretty good at interviews. It turns out that they try to assess soft skills with technology questions based on the luck- of- the- draw of candidate resumes. Candidates with effective resumes will have an easy time passing these interviews. From the comments on this thread, that obviously sounds reasonable to some people.

I submit: those people are not competing for talent. They may think they are, but the real contenders in this market won't be OK with letting good candidates slip past because their job-hunting skills aren't finely tuned. In fact: they'll do the opposite: those candidates are steals in this market.

The original commenter acknowledged that when he said that his experience was that the market was full of poor candidates. If that's the case, you especially can't afford to filter out effective devs because they fail to impress you when they explain how they use version control, or when their resume overstates their facility with version control.

Re: The Terrible Technical Interview

#202

Post by palimpsests » Sun, Mar 22, 2015, 4:28 PM UTC

sanderjd wrote: post #99

guelo wrote: post #30

Earlier quoted context omitted.

You should spend time improving your live performance skills. It comes in handy for more than just job interviews.

I think you're correct, but I also think it's a real shame that live performance skills are important to excel at a career that has nothing to do with live performance. I wonder if there are any careers where you only have to get good at the thing you're doing rather than a bunch of meta-things.

Live performance skills are most absolutely unnecessary to excel in development; this comment thread is discussing how certain people can improve their interviewing performance through the practice of being spontaneous and present. A very helpful trait that some people have naturally, and that others do not.

I think it's great when the practice of some discipline or craft has beneficial effects in other areas of our our lives.

Re: The Terrible Technical Interview

#203

Post by tptacek » Sun, Mar 22, 2015, 4:39 PM UTC

pmiller2 wrote: post #112

tptacek wrote: post #75

Earlier quoted context omitted.

I like this summary and plan on stealing it from you.

@tptacek, I'd be interested in your thoughts on how one could adapt your methods from Matasano to, say, a startup doing web dev or to an enterprise software company. I've done some thinking based on some of your writings and the idea that the best way to test someone for suitability to do a job is to have them do the job, and haven't really come up with any good ideas.

1. Take application you've already built.

2. Package it, with all of the assets and utilities needed to get it running with "vagrant up".

3. Carve out some feature/features from the application, and replace them with stub functionality.

4. Deliver the vagrant app and a functional spec to candidates. Have them implement the missing feature.

5. Devise a scoring rubric (unit test coverage, lines of code, algorithms used, safe/unsafe APIs, performance, whatever). Mechanically evaluate candidate submissions.

6. (Optional) Devise a 15-20 minute on-site interview component to verify that the candidate actually did the work. We didn't bother with this, and multiplied the size of our team (NCC is the largest software security firm in North America) and had 100% retention. But it's a big concern for some people.

Re: The Terrible Technical Interview

#204

Post by fsk » Sun, Mar 22, 2015, 4:52 PM UTC

din-9 wrote: post #173

fsk wrote: post #56

Earlier quoted context omitted.

Also, what makes this mistake common is the feedback you get. When you make a false negative, you never find out that you passed on someone amazing. When you make a false positive, it's professional embarrassment for the boss when he's forced to admit he made a mistake and fire them. So the incentive for the boss is to minimize false positives, even at the expense of too many false negatives. The boss is looking out…

Reasons for not wanting to fire people also include: basic humanity.

But it's inhumane to your other employees to keep someone unqualified around.

Re: The Terrible Technical Interview

#205

Post by SnacksOnAPlane » Sun, Mar 22, 2015, 4:55 PM UTC

waterlesscloud wrote: post #83

SnacksOnAPlane wrote: post #73

Earlier quoted context omitted.

Why is Atlanta not attractive to tech workers? Georgia Tech has a phenomenal CS program, and we've got a pretty large and diverse startup population. I'm not sure if what you're saying isn't true, or if I'm just biased and not able to see Atlanta from an outsider's perspective.

Atlanta is a pleasant enough city. I'm from there originally, though I moved to L.A. about 15 years ago. I left because I wanted to live in a big city, not in suburbia. Atlanta has some of the plusses of a city, but they're relatively limited compared to LA, SF, NY, DC. It's more a collection of suburbs than a city. If suburban life is what you want, Atlanta is fine for that. Honestly, most of Silicon Valley is subur…

You should come back sometime! The Beltline has really reinvigorated a lot of intown areas. Midtown is booming, as is the Old Fourth Ward and lots of other intown neighborhoods.

But I do get your point about "odd" lifestyles. We're certainly getting weirder here, but I don't think it can yet compare to what you see in NYC or the Bay Area. Still, there's a lot to like here, and cost of living alone is a great argument for Atlanta.

Re: The Terrible Technical Interview

#206

Post by tedunangst » Sun, Mar 22, 2015, 5:01 PM UTC

frostmatthew wrote: post #188

tedunangst wrote: post #157

Earlier quoted context omitted.

Consider the set ( -10, 4, 4, 4, 4, 4, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 20, 50). It has 20 elements and an average of 10. 5% are toxic. 25% are below average. 60% are average. 10% are above average.

> Consider the set ( -10, 4, 4, 4, 4, 4, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 20, 50). I didn't say it was impossible to construct a set that would yield only 10% as above average, I said there "is no reasonable definition of average" - if you feel the above set accurately represents the distribution of the caliber of developers then we clearly have very different opinions of what's "reasonable."

I don't think it's too far off the mark. What do you think the distribution looks like?

Or one could replace the mathematical term average with the word ordinary. Is it possible for 60% of developers to be ordinary?

Re: The Terrible Technical Interview

#207

Post by nitrogen » Sun, Mar 22, 2015, 5:05 PM UTC

mtviewdave wrote: post #33

StevePerkins wrote: post #6

I do almost all of the technical phone screens for my company. The process starts with a "where-do-you-see-yourself-in-5-years" personality screen, with an H.R. drone or outside recruiter. It moves on me, and then on to a brief "homework" coding exercise that we ask people to write and submit. If we like their code sample, then they come in for the panel of "whiteboard-exercise" people who conduct the face to face ro…

> "I see that you've spent X years using Subversion for source control. What are your opinions on trunk-first development vs. branch-first development?" Actually, this demonstrates a problem right here. It's not clear to me what you mean by these terms. A quick Google search ("trunk-first vs branch-first svn") suggests that you're not using common terminology. After thinking about it for a minute or two, what I _thin…

A quick Google search ("trunk-first vs branch-first svn") suggests that you're not using common terminology.

It's not a terminology question, it's a concept question. A good candidate should be able to recognize abstract concepts regardless of the words used to describe them. Even in an interview setting.

Re: The Terrible Technical Interview

#208

Post by tptacek » Sun, Mar 22, 2015, 5:19 PM UTC

facepalm wrote: post #190

tptacek wrote: post #52

Requiring a side project from candidates would have cost us most of our best hires.

But I think you advocate doing a project with the candidate, which is also very time consuming. Doesn't that also filter out some good candidates who simply don't have the time? I assume you pay them, but still. After a couple of job interviews in recent times my personal inclination to invest a lot of time has gone down by a huge amount. For my first application I even took the time to contribute to one of their ope…

No, we did not ask candidates to do a side project for us. All we did was move time they would have spent either on the phone or in a face-to-face interview to something they could instead do at home.

Re: The Terrible Technical Interview

#209

Post by tptacek » Sun, Mar 22, 2015, 5:21 PM UTC

nitrogen wrote: post #207

mtviewdave wrote: post #33

Earlier quoted context omitted.

> "I see that you've spent X years using Subversion for source control. What are your opinions on trunk-first development vs. branch-first development?" Actually, this demonstrates a problem right here. It's not clear to me what you mean by these terms. A quick Google search ("trunk-first vs branch-first svn") suggests that you're not using common terminology. After thinking about it for a minute or two, what I _thin…

A quick Google search ("trunk-first vs branch-first svn") suggests that you're not using common terminology. It's not a terminology question, it's a concept question. A good candidate should be able to recognize abstract concepts regardless of the words used to describe them. Even in an interview setting.

Why? What makes being able to persuasively discuss this particular concept an attribute of a good candidate? And, when you identify the answer to that question, can you then answer: is this question the best means I have of assessing that attribute?

Re: The Terrible Technical Interview

#210

Post by mechanical_fish » Sun, Mar 22, 2015, 5:46 PM UTC

tedunangst wrote: post #166

mechanical_fish wrote: post #165

Earlier quoted context omitted.

It's highly relevant to getting a job on a software team, but only because most teams are assembled via interviews, and verbally relating stories from your past in a face-to-face meeting is the most effective interviewing technique. And it's relevant to a job like sales. (That's unsurprising because a job interview is actually a sales meeting.) And many management jobs do have a sales aspect -- you have to justify bu…

I would say the whole point of software is to build something that people can use. If someone builds a module, but can't tell me how to use it, it's not of much use to me. Oh, but surely they'll write documentation? Programmers, by and large, seem to suck at that too, which is why we have tech writers. But somebody still needs to explain to the tech writer what's happening! And only a few companies seem to carry tech…

Your questions are good and relevant ones, I agree. Let me rephrase them a little:

"Hello, coworker! Did you enjoy the cake we both got to eat the other day in the company cafeteria?"

"By the way, I'd like to ask you a question, and don't worry: This isn't an interview or anything, so if you can't answer me right away, or if your answer lacks grace, it's not as if you'll lose your job."

"Anyway, coworker: I found this code, which you wrote while working for my company, under the direction of my company's management, and which solves a problem that my company actually has, and which builds upon my team's platforms, languages, and coding standards, and which might even link directly to my code, and which both of us have had a moment to read and think about and which is right in front of us on this monitor. How does this code work?"

"Also, can you help me debug this code I have here? It builds atop the code I showed you last week, and is written in the same language that we all use, and attempts to solve a problem you've seen before – which is not a coincidence, because you were the person who asked me to solve the problem."

These questions are incredibly relevant to our work, but interviews can't cover them. Candidates are not our coworkers and they share none of our context. Instead, interviews are, at best, an exercise in prediction. In practice, they are often an exercise in magical thinking.

During the workday, people aren't being constantly judged. They don't implement functions on whiteboards without unit tests, solve brainteasers out loud during stand-up meetings, or implement quicksort from memory. They do have to explain code to coworkers, but not to people who don't understand the problem space, the language, the background, or the constraints. These rarely-exercised feats of skill are valuable -- sales is valuable -- and our gut feeling is that such feats are somehow related to relevant job skills. But gut feelings are often wrong. And not every job is in sales.