Three hundred programming interviews in thirty days • Hacker News Board

Re: Three hundred programming interviews in thirty days

#141

Post by mVChr » Tue, Jun 23, 2015, 9:55 PM UTC

m3mnoch wrote: post #136

while this process feels like it's hitting the sweet spot for finding out who can write brilliant code, that's half or less of the battle in hiring people. personally (and as a hiring manager), i feel like a majority of the hiring process is dependent (obviously) on the environment you're hiring into. hiring for that small startup? you'll want multi-hat wearing people first, brilliant programmers second. hiring for a…

Hear hear, the "no assholes" rule is always good to apply, regardless of programmer brilliance.

Re: Three hundred programming interviews in thirty days

#142

Post by Akkuma » Tue, Jun 23, 2015, 9:57 PM UTC

Naomarik wrote: post #29

I got a phone interview with Harj and wasn't considered for anything further. I'm not sure what kind of hackers they were looking for, but I've been directly involved with creating the infrastructure used in marketing campaigns with the likes of CNN, McDonalds, Infiniti, and more. I've turned an idea into a company with 8 full time employees and have investors seriously interested in one of my side projects. I'm curr…

I made it through the phone interview and honestly, you sound like a better candidate than me. Momma always said I should be a lawyer.

I have 8 years experience and talked about my personal minimal JavaScript framework I built.

I didn't pass the next interview, which in my opinion their provided reasoning was kind of atrocious. In fact, I got the "wrong" feedback sent to me, which for the most part I feel I refuted, only to then get a follow up that basically they sent the wrong feedback and I fiddled around to get things working too much for their liking, despite ultimately getting them both running.

Re: Three hundred programming interviews in thirty days

#143

Post by el_fuser » Tue, Jun 23, 2015, 9:57 PM UTC

KingMob wrote: post #55

ammon wrote: post #15

Earlier quoted context omitted.

Hey! Author here. Yes, we did not know that the screening steps were meaningful, so for the first 300 applicants, we interviewed everyone, even people who performed badly. We then looked for correlation between screening step scores, and programming interview results. Doing well on the fizzbuzz problems was not very correlated. By dropoff I mean people who left during a step, and never logged back into our site.

I appreciate that you guys might not be statisticians, but if you're going to try and analyze data like this, you simply must address survival bias. As it stands, these data are meaningless unless you assume dropouts are completely unrelated to your screening. You claim doing well on the Fizzbuzz wasn't correlated with interview performance, but you also said "We saw twice the drop off rate on the coding problems as…

>> I appreciate that you guys might not be statisticians

and apparently very few on HN in general, seeing how this piece was upvoted.

Google has been studying their process in the last few years and are reaching different conclusions than these guys.

Leave it to the reader to ascertain which group has more validity.

Re: Three hundred programming interviews in thirty days

#144

Post by rezistik » Tue, Jun 23, 2015, 9:59 PM UTC

cogg wrote: post #75

rezistik wrote: post #33

Earlier quoted context omitted.

I personally prefer it to a point. My only request when I get this type of interview is that I'm allowed to make the project and requirements public on my github for future interviewers. If I'm doing a free project I feel it's fair.

I am curious why you have that requirement. What is the fairness you are striving for?

A lot of companies will ask for existing code examples, in this way I can just point them to my example-code github repo and I've had companies use that instead of asking me to do a specific challenge for them.

Re: Three hundred programming interviews in thirty days

#145

Post by ionforce » Tue, Jun 23, 2015, 10:07 PM UTC

mVChr wrote: post #137

geebee wrote: post #39

Earlier quoted context omitted.

I did this once. After a take home quiz that took about an hour, I had a 6-7 hour "homework" problem. I did it, and sent it in. It took the company a month to get back to me, though I did stay in contact with the recruiter. The final answer, delivered by the recruiter, was "we've decided not to more forward at this time." No other feedback. I have no idea if anyone even looked at it - I never did speak to a dev at th…

One take-home project I was given was very cleverly designed. Without going into too much detail, the project description initially seemed pretty simple. The actual getting-it-to-work part took about half an hour. However there was a requirement near the end that said the whole operation had to complete in less than X amount of time. The initial straightforward implementation I wrote took about 8-times longer to comp…

Similar to how Project Euler works.

There are naive ways to solve certain problems (finding primes), but there is a general guideline that tasks should finish in under one minute (I believe).

Re: Three hundred programming interviews in thirty days

#146

Post by gjm11 » Tue, Jun 23, 2015, 10:10 PM UTC

foobarqux wrote: post #91

overgryphon wrote: post #77

Earlier quoted context omitted.

Interview environments are always stressful, but so are other common workplace situations. Being unable to manage stress effectively might contribute to poor on-the-job performance, even for people with great pure coding ability. The process right now seems focused on finding people who are the best at only coding. In the future, do you intend to also consider communication skills? Senior developers act as mentors fo…

Interview stress is of the type I call "defusing the bomb": everything at stake, essentially immediate time constraints and confrontational. This never occurs in real life.

"Unfortunately something not so different sometimes does. E.g., you're at a startup, you have a critical demo for your best-hope customer, it was scheduled very aggressively because the CEO wanted to fit the customer's availability, it's happening in one hour, and nothing is working.

Not exactly confrontational (though it could easily get that way if you aren't careful) but just as stressful.

Of course it's far better to avoid such situations, but they do happen.

Re: Three hundred programming interviews in thirty days

#147

Post by mVChr » Tue, Jun 23, 2015, 10:11 PM UTC

choppaface wrote: post #17

I've done over 1000 interviews and my experience agrees with their findings that talking about a project is not a good predictor of coding. I usually left detailed resume questions for the end since so many candidates would bomb the coding part of the interview. I'm surprised they didn't get stronger results from fizz buzz, but I noticed among the candidates I saw that the percentage of 'non-coders' is substantial bu…

The only time I've found talking about a project to be a good predictor is when it's a type of project or tech stack that I'm familiar with. In that case, I can ask very specific probing questions about problems I'd encountered with that problem or tech. In answering these questions whether and how often they talk about specific solutions they came up with versus saying something along the lines of "well I dunno, I wasn't responsible for that piece" or "I'm not sure about the details, I was more involved with the higher level decision making" can be very telling.

Re: Three hundred programming interviews in thirty days

#148

Post by brudgers » Tue, Jun 23, 2015, 10:12 PM UTC

Retra wrote: post #126

brudgers wrote: post #78

Earlier quoted context omitted.

Professional licencing processes establish minimum standards of competence, training and education. They do not guarantee excellence and a license represents that its holder is no worse than the worst legally allowable practitioner. Half of all surgeons are below average. You don't want one of them if you have a choice.

Half are below the average of practicing surgeons. But the average surgeon is much better because you weed out a lot of incompetent people who can't get certified.

Surgeons are not number 10 heating oil commodities. The difference in quality between one surgeon and the next matters when it comes to an operation's outcome. Likewise - and this is the gist of my comment - licensing software engineers might raise the minimum bar, but it won't make hiring any easier because licensing doesn't establish relevant expertise or pertinent experience. Obstetricians not thoracic surgeons are what you want for Cesareans just as you don't want a J programmer writing your CMS.

Re: Three hundred programming interviews in thirty days

#149

Post by acbart » Tue, Jun 23, 2015, 10:13 PM UTC

ammon wrote: post #79

mbesto wrote: post #19

> In our first 30 days, we've come up with a replacement for resume screens, and shown that it works well. What's the metric that shows it works well? > The really exciting point comes when we can re-run all this analysis, basing it on actual job performance, rather than interview results. And how precisely do you measure job performance? If this is achievable, I've got a line of companies out my door that would love…

Measuring job performance is hard. A good analogy is measuring intelligence. The IQ test is clearly bullshit (it does not come close to reflecting the complexity and high dimensionality of human intelligence). That said, it's a useful research tool. Across a large number of people in aggregate, it can be used to learn things (like that leaded gas was a really bad idea). We'll be doing the same sort of (very imperfect…

I'll be very interested in hearing what your measures are, and how you'll account for things like employees being in a bad environment/having bad bosses. It's a complicated thing to measure.

Are you guys planning on publishing any of this material in a journal or research venue, or will you keep the results to blog posts?

Re: Three hundred programming interviews in thirty days

#150

Post by jqm » Tue, Jun 23, 2015, 10:24 PM UTC

"suddenly, a significant percentage of the people who had spoken well about impressive-sounding projects failed, in some cases spectacularly, when given relatively simple programming tasks."

Unsurprising. Turns out talk really is cheap and doesn't indicate one can do. I've even seen people able to maintain jobs over a period of years by talk alone without ever really doing much. Or even being able to do much.