Three hundred programming interviews in thirty days • Hacker News Board

Re: Three hundred programming interviews in thirty days

#121

Post by avodonosov » Tue, Jun 23, 2015, 9:10 PM UTC

lackbeard wrote: post #35

Most candidates can speak well about interesting projects they'd worked on in the past, but a significant percentage of those can't pass a coding test. Also, not being able to speak well about a past project is highly correlated with doing well on a coding test. Sounds like one or the other should be thrown out. (Or maybe only the small percentage who do well on both will go on to do well on the job?)

One or other should be thrown out. Or both thrown out. Or neither thrown out.

Maybe only the small percentage who do well on both will go on to do well on the job. Or maybe large percentage will do well on the job. Or maybe a percentage who do bad on both will do well on the job.

In short: there is no data at all about doing well on the job, so we don't know what to throw out and whom to chose.

Re: Three hundred programming interviews in thirty days

#122

Post by ruswick » Tue, Jun 23, 2015, 9:12 PM UTC

mseebach wrote: post #119

protonfish wrote: post #6

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of giving a candidate a project to do "in their own time." I recently had a interview that included this and I can share the result: I accepted an offer from a different company that didn't require it. I doubt my life is that different than anyone else's, with a fu…

I'm sure it's true that a non-trivial number of good candidates will reject a job opportunity when asked to do a work sample. These, however, will generally be those with plenty of attractive alternative offers - the ones well known in the community, with a good network etc. At the other end of the stick, there are those still trying to break in to the field. They may have moved to a new city, they may have worked in…

It sounds like your process went ok. But one hour tests are one thing. Projects that may take 10 or more hours are totally different. It really is amazing what some of these companies will demand of their applicants before even granting an interview.

And although this system might be more accessible than traditional recruiting, it is far from perfect. It is the job of the community to demand better and more respectful hiring practices from companies. Paying the applicant for the time they spent working on the project or offering a traditional interview for people who work full time and have families would be a good start.

Re: Three hundred programming interviews in thirty days

#123

Post by vorg » Tue, Jun 23, 2015, 9:15 PM UTC

brudgers wrote: post #66

Buetol wrote: post #47

Earlier quoted context omitted.

A simple solution is for the company to offer you something for the time you spent doing the sample test (money, gift card, their product,...)

How about a car? Then it is obviously worth the applicant's time and since a professional can deliver or destroy a Bimmer's worth of corporate value on any given decision it easily pays for itself in identifying good candidates and weeding out bad ones.

> it easily pays for itself in identifying good candidates and weeding out bad ones

It even easily pays for itself in just weeding out the bad ones.

Re: Three hundred programming interviews in thirty days

#124

Post by logfromblammo » Tue, Jun 23, 2015, 9:18 PM UTC

s0uthPaw88 wrote: post #73

protonfish wrote: post #6

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of giving a candidate a project to do "in their own time." I recently had a interview that included this and I can share the result: I accepted an offer from a different company that didn't require it. I doubt my life is that different than anyone else's, with a fu…

I feel that if you are asking a company to invest $100k/yr on you, it is not unreasonable for them to ask for a few hours of your time.

It's not unreasonable for the company to ask for a few hours of the person they actually end up hiring. It's wasting the time of the dozens of other people that will never get reasonable consideration that's objectionable.

Wasting candidate time should never be used for screening. At most, steps that take a significant amount of time or effort should be reserved solely for determining the order in which the acceptable candidates receive offers, and the amounts of those offers.

Re: Three hundred programming interviews in thirty days

#125

Post by Kapow » Tue, Jun 23, 2015, 9:21 PM UTC

brudgers wrote: post #78

gwbas1c wrote: post #63

Many fields require serious professional certification. You can't become a doctor unless you go through a board certification that includes simulated patient interaction. Likewise, you can't build a bridge or a dam without engineering certification. IMO, this article demonstrates the need to certify software engineers; using a process similar to the interviewing process described. Therefore, when hiring, we can skip…

Professional licencing processes establish minimum standards of competence, training and education. They do not guarantee excellence and a license represents that its holder is no worse than the worst legally allowable practitioner. Half of all surgeons are below average. You don't want one of them if you have a choice.

And if you had a choice between a below-average licensed surgeon and an unlicensed surgeon?

There's no process that can guarantee above-average practitioners for everyone, it's a paradox. That doesn't mean minimum standards are pointless.

Re: Three hundred programming interviews in thirty days

#126

Post by Retra » Tue, Jun 23, 2015, 9:21 PM UTC

brudgers wrote: post #78

gwbas1c wrote: post #63

Many fields require serious professional certification. You can't become a doctor unless you go through a board certification that includes simulated patient interaction. Likewise, you can't build a bridge or a dam without engineering certification. IMO, this article demonstrates the need to certify software engineers; using a process similar to the interviewing process described. Therefore, when hiring, we can skip…

Professional licencing processes establish minimum standards of competence, training and education. They do not guarantee excellence and a license represents that its holder is no worse than the worst legally allowable practitioner. Half of all surgeons are below average. You don't want one of them if you have a choice.

Half are below the average of practicing surgeons. But the average surgeon is much better because you weed out a lot of incompetent people who can't get certified.

Re: Three hundred programming interviews in thirty days

#127

Post by Silhouette » Tue, Jun 23, 2015, 9:22 PM UTC

s0uthPaw88 wrote: post #73

protonfish wrote: post #6

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of giving a candidate a project to do "in their own time." I recently had a interview that included this and I can share the result: I accepted an offer from a different company that didn't require it. I doubt my life is that different than anyone else's, with a fu…

I feel that if you are asking a company to invest $100k/yr on you, it is not unreasonable for them to ask for a few hours of your time.

And if you're worth $100k/year and they're asking for a day of your time, it's not unreasonable for you to ask for $500 from them for turning up. How many employers do you know who offer that to all of their interview candidates?

Re: Three hundred programming interviews in thirty days

#128

Post by rudolf0 » Tue, Jun 23, 2015, 9:22 PM UTC

ammon wrote: post #79

mbesto wrote: post #19

> In our first 30 days, we've come up with a replacement for resume screens, and shown that it works well. What's the metric that shows it works well? > The really exciting point comes when we can re-run all this analysis, basing it on actual job performance, rather than interview results. And how precisely do you measure job performance? If this is achievable, I've got a line of companies out my door that would love…

Measuring job performance is hard. A good analogy is measuring intelligence. The IQ test is clearly bullshit (it does not come close to reflecting the complexity and high dimensionality of human intelligence). That said, it's a useful research tool. Across a large number of people in aggregate, it can be used to learn things (like that leaded gas was a really bad idea). We'll be doing the same sort of (very imperfect…

That is an excellent analogy. (Not being sarcastic.)

Re: Three hundred programming interviews in thirty days

#129

Post by nvarsj » Tue, Jun 23, 2015, 9:22 PM UTC

geebee wrote: post #39

protonfish wrote: post #6

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of giving a candidate a project to do "in their own time." I recently had a interview that included this and I can share the result: I accepted an offer from a different company that didn't require it. I doubt my life is that different than anyone else's, with a fu…

I did this once. After a take home quiz that took about an hour, I had a 6-7 hour "homework" problem. I did it, and sent it in. It took the company a month to get back to me, though I did stay in contact with the recruiter. The final answer, delivered by the recruiter, was "we've decided not to more forward at this time." No other feedback. I have no idea if anyone even looked at it - I never did speak to a dev at th…

I like the idea of giving candidates a choice. For me personally, I would almost always take the "homework" if it meant no (or very short) in person interview. I like to code, not interview. Still, 6-7 hours is pushing it, it should be a few hours (3 or 4 at max, if you're not going to do a face to face), and a genuinely interesting problem (the kind of problem can tell you a lot about the company itself!).

When I interviewed at Apple years ago, they gave me a take home program to solve. It was interesting and I enjoyed it a lot. It required some good algo knowledge and had to be written in C. I was told in my face to face that few people ever solve it, and I enjoyed discussing it.

But the rest of the interview was unpleasant - your typical SV brain teaser riddled, algo whiteboard sessions for 4 hours, 2 days in a row. I wasn't really expecting it and wasn't prepared (my fault), so failed miserably. I think I suffered PTSD after that for a while :).

Now I feel preparing for that sort of interview is a huge waste of my own time. It has so little relevance to reality. I'd much rather put that time into building a product or learning some new PL or whatever. At least with a coding test I'm honing skills that I'll use day to day.

Re: Three hundred programming interviews in thirty days

#130

Post by rezistik » Tue, Jun 23, 2015, 9:23 PM UTC

ammon wrote: post #46

rezistik wrote: post #33

Earlier quoted context omitted.

I personally prefer it to a point. My only request when I get this type of interview is that I'm allowed to make the project and requirements public on my github for future interviewers. If I'm doing a free project I feel it's fair.

Well, that's hard. Standardizing evaluation (critical to reducing bias) requires that everyone (or a lot of people) see a similar problem. And when someone is working on their own time, I think it's important that there's not working code easily accessible online. Perhaps this can be overcome by asking candidates to talk through their code after they are done

In most of the work at home projects I've been given there was a ton of available code online, they've primarily been for front-end positions so it's usually "Make a small app using your favorite library"

I can't imagine there are that many 3-4 hour projects that won't already have available code. The most important part of an at home project in my opinion is the walk through.

The best interview I had asked me to go through the project and explain the what and why I did things and asked for high level understanding of what the framework I chose or the browser/node was doing based on what I wrote.

It was an app in React and they went through why I decided to make a component for x, y but not z and then asked if I understood the virtual DOM as a concept.

To me that's the best you can do, you need to understand how the programmer thinks about problems, works through them and that they are engaged in the ecosystem at large.