Three hundred programming interviews in thirty days • Hacker News Board

Re: Three hundred programming interviews in thirty days

#161

Post by bobbles » Tue, Jun 23, 2015, 11:25 PM UTC

geebee wrote: post #39

protonfish wrote: post #6

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of giving a candidate a project to do "in their own time." I recently had a interview that included this and I can share the result: I accepted an offer from a different company that didn't require it. I doubt my life is that different than anyone else's, with a fu…

I did this once. After a take home quiz that took about an hour, I had a 6-7 hour "homework" problem. I did it, and sent it in. It took the company a month to get back to me, though I did stay in contact with the recruiter. The final answer, delivered by the recruiter, was "we've decided not to more forward at this time." No other feedback. I have no idea if anyone even looked at it - I never did speak to a dev at th…

I wish experiences like these were put on glassdoor as well, it'd be good to know what their hiring process is like rather than just actually working there.

Maybe enough responses about never getting contact from employers would make them start treating people like human beings

Re: Three hundred programming interviews in thirty days

#162

Post by bjt » Tue, Jun 23, 2015, 11:25 PM UTC

It's very nice to see hiring advice based on data rather than anecdotes. But I wonder if the process described in the article is pre-selecting for people who are out of work and desperate, rather than currently employed and casually looking for something better.

From the article: > Our process has four steps:

> 1. Online technical screen.

> 2. 15-minute phone call discussing a technical project.

> 3. 45-minute screen share interview where the candidate writes code.

> 4. 2-hour screen share where they do a larger coding project.

Then later:

> ...we can't afford to send people we're unsure about to companies

Does every applicant in this system really have to go through four rounds of screening before even talking to someone who works at the actual company? I can't imagine doing that unless I was desperate.

Re: Three hundred programming interviews in thirty days

#163

Post by senorprogrammer » Tue, Jun 23, 2015, 11:26 PM UTC

Where I work, we ask candidates to do a take-home task after they've passed the first in-person interview. The take-home task is trivial, something most developers with a few years experience should be able to knock out in less than an hour. All components of it are easily googleable.

The advantage is that we're building a corpus of solutions to the same problem that we can compare against each other, which is interesting. More importantly, we're building a corpus of solutions that we can then pick from to have the candidate analyze in-person, and talk us through what they see, what they'd do differently, what they like/don't like, etc.

In short, we familiarize them with the problem via their own answer, and then ask them to analyze someone else's (anonymized) answer. Our sample set so far is too small to draw definitive conclusions from, but it feels better than our old ways of doing things.

Re: Three hundred programming interviews in thirty days

#164

Post by gjm11 » Tue, Jun 23, 2015, 11:28 PM UTC

RogerL wrote: post #153

gjm11 wrote: post #146

Earlier quoted context omitted.

"Unfortunately something not so different sometimes does. E.g., you're at a startup, you have a critical demo for your best-hope customer, it was scheduled very aggressively because the CEO wanted to fit the customer's availability, it's happening in one hour, and nothing is working. Not exactly confrontational (though it could easily get that way if you aren't careful) but just as stressful. Of course it's far bette…

You are ignoring the data. Person after person says they do fine in the job, yet have trouble with interviews. I've presented in rooms where the lowest ranking officer was a colonel, and most were important people at the Pentagon. No freeze up, easy peasy, because I know what the hell I'm talking about and because I know I'm not going to be judged on some bullshit evaluation. (re: "x Don't make up theoretical situati…

What data am I ignoring, and how do you know?

I made no comment on what kind of interview works best. I merely observed -- what is certainly true -- that sometimes "defusing the bomb" situations actually do come up in real life, where everything is at stake and you have to work under very severe time pressure.

Do you disagree that such situations sometimes arise in real life?

If not -- if your point is that that isn't necessarily a good reason for interviews to involve such situations -- then I think we are in violent agreement. I am not arguing for defusing-the-bomb interviews. I just don't think it's quite right to say that in real life you never have to defuse a bomb.

Re: Three hundred programming interviews in thirty days

#165

Post by x0x0 » Tue, Jun 23, 2015, 11:29 PM UTC

geebee wrote: post #39

protonfish wrote: post #6

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of giving a candidate a project to do "in their own time." I recently had a interview that included this and I can share the result: I accepted an offer from a different company that didn't require it. I doubt my life is that different than anyone else's, with a fu…

I did this once. After a take home quiz that took about an hour, I had a 6-7 hour "homework" problem. I did it, and sent it in. It took the company a month to get back to me, though I did stay in contact with the recruiter. The final answer, delivered by the recruiter, was "we've decided not to more forward at this time." No other feedback. I have no idea if anyone even looked at it - I never did speak to a dev at th…

It's really amazing, isn't it?

I talked to thumbtack for data science. They wanted a 10+ hour takehome project after just speaking to a recruiter because their data scientists were "too busy" to speak to candidates.

Yeah, I'm going to get right on that.

Bet they're bitching about a data scientist "shortage (^1)" as we speak.

(1) data scientist defined as experienced data scientists who want to waste a day's labor before even understanding what they'd be working on and/or speaking to their potential boss

edit: checked my email archive and fixed company name. Apologies...

Re: Three hundred programming interviews in thirty days

#166

Post by curun1r » Tue, Jun 23, 2015, 11:30 PM UTC

ammon wrote: post #15

jasode wrote: post #4

Imo, this post did not organize its data and findings into a coherent presentation. For example... >The fizzbuzz-style coding problems, however, did not perform as well. While the confidence intervals are large, the current data shows less correlation with interview results. [...] The coding problems were also harder for people to finish. We saw twice the drop off rate on the coding problems as we saw on the quiz. I…

Hey! Author here. Yes, we did not know that the screening steps were meaningful, so for the first 300 applicants, we interviewed everyone, even people who performed badly. We then looked for correlation between screening step scores, and programming interview results. Doing well on the fizzbuzz problems was not very correlated. By dropoff I mean people who left during a step, and never logged back into our site.

Asking the author because I'm curious...

Are you tracking longer-term hiring outcomes too? They'll probably take some time to become meaningful, but they're far more important. The data you've compiled is useful since it helps to filter earlier in the process, but it still presumes that your in-person interviewing process makes the correct decision. If the final filter is letting bad candidates through or screening out good candidates, all the correlations you've found could be reflecting only the ability to pass the interview, not the ability to do the job successfully.

Hopefully you're continuing to follow hires 1, 2, 5 years after being hired to tie it back to the data you collect about the interview process. It would be awesome if you could find predictors of candidates that are likely to quit less than a year after being hired or candidates that will receive less-than-stellar ratings from their managers. By doing this, you'd help hiring managers deal with the blindspots in their hiring, not just streamline the existing process.

Re: Three hundred programming interviews in thirty days

#167

Post by panic » Tue, Jun 23, 2015, 11:38 PM UTC

mwfunk wrote: post #152

Preamble: not middlebrow dismissal, I really like what they're doing here and will pay attention to them going forward. This is just picking a nit that my hypersensitive self just can't resist: "Fizz buzz style coding problems are less predictive of ability to do well in a programming interview" I'm sure this is 100% true, but I thought the point of fizzbuzz-type problems were to weed out people who couldn't program…

It would be interesting to separate out the sensitivity (how often does the test give a positive result on good candidates?) from specificity (how often does the test give a negative result on bad candidates?) for each of these tests. I'd guess a fizz-buzz-style test has good sensitivity but terrible specificity.

Re: Three hundred programming interviews in thirty days

#168

Post by [deleted] » Tue, Jun 23, 2015, 11:40 PM UTC

mVChr wrote: post #141

m3mnoch wrote: post #136

while this process feels like it's hitting the sweet spot for finding out who can write brilliant code, that's half or less of the battle in hiring people. personally (and as a hiring manager), i feel like a majority of the hiring process is dependent (obviously) on the environment you're hiring into. hiring for that small startup? you'll want multi-hat wearing people first, brilliant programmers second. hiring for a…

Hear hear, the "no assholes" rule is always good to apply, regardless of programmer brilliance.

[deleted]

Re: Three hundred programming interviews in thirty days

#169

Post by tomtang0514 » Tue, Jun 23, 2015, 11:49 PM UTC

lockes5hadow wrote: post #133

tomtang0514 wrote: post #120

Earlier quoted context omitted.

I've seen several companies giving out such homework to me or my friends back in college. And to be honest, none of the are among the "great culture company list" people are talking about. Beside all the issues you mentioned above, those tests don't have a clear PASS line. You spent hours on that and can be rejected for no reason. Back in college I've done a code challenge from Box and my solution ranked No.1 in thei…

Even in C I would point out that the compiler will likely make such a trivial optimization for you. That was clearly a company full of morons.

Regardless of the compiler, let's say (x Some interviewers (not just the "homework") raise their bar by not allowing you to make any tiny mistake. And I just don't get it. If someone is good enough to write something like a lite version of Hacker New website in hours, I'm not going to turn him/her down because of such mistake.

Re: Three hundred programming interviews in thirty days

#170

Post by seajosh » Tue, Jun 23, 2015, 11:50 PM UTC

I personally hate the "take home test" approach to interviewing. I've had multiple such tests that take anywhere from 10-25 hours to complete because simply answering the question isn't enough; you need to give textbook correct answers and your code must be formatted perfectly with the requisite comments and documentation. In short, it's pretty similar to an upper-level college course's final exam; however, in college, you can get a good grade with a few mistakes; in interviewing, you get rejected for a few mistakes. I'm done giving a company 15 hours of my time just to get to a first interview; this is arrogant, condescending, and completely devalues my time.

The reality of hiring is you're going to make mistakes, like every other part of running a business. Even in an extended "interview" such as dating for a potential life partner, people make mistakes so I'm not sure how the hiring process can be quantified to remove said error. The interview process is so excruciating these days I often hate the companies I'm talking with.

While we're at it, the skills requirements listed with jobs today are astounding. My experience is that a company wants to hire a programmer with at least a journeymen's level of expertise in 6-8 skills. If you have 5 and are comfortable you can learn the other 3, you're dead in the water. Let's be honest, the latest Javascript framework isn't that complicated. The latest NoSQL database isn't that hard to learn.

The truly hard parts of joining a new company are learning how projects are managed, getting the political lay of the land, finding a sherpa to answer your questions in the first couple of weeks, and learning where you fit within the organization.