Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Three hundred programming interviews in thirty days

blog.triplebyte.com

Post reply on HN
Print view View original

41–50 of 248 posts

Re: Three hundred programming interviews in thirty days

#41

Post by jbob2000 »

ammon wrote: post #27
protonfish wrote: post #6

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of giving a candidate a project to do "in their own time." I recently had a interview that included this and I can share the result: I accepted an offer from a different company that didn't require it. I doubt my life is that different than anyone else's, with a fu…

Yeah, we agree. We don't plan on asking everyone to do hours of work on their own. A lot of people don't have the time. However, we're also seeing a bunch of applicants who are good programmers, but become really stressed in interviews and do badly (they freeze). These people have trouble getting jobs. What we want to do is offer them the option of doing a larger project on their own, rather than our final interview.…

Don't most companies have probation periods? At my company, we have a really easy interview. It's really just a check to make sure you exist, don't have large red flags, and can show up on time. No technical tests, just some light conversation.

Then we have a 3 month probation period. This is the "real test". Can you write code? Can you communicate with the team? Do you fit in with the culture? Etc.

When we get a bad hire, they usually recognize that they don't fit or aren't working and end up leaving of their own volition. Rarely do we actually have to fire someone.

Re: Three hundred programming interviews in thirty days

#42

Post by ddrum001 »

ammon wrote: post #27
protonfish wrote: post #6

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of giving a candidate a project to do "in their own time." I recently had a interview that included this and I can share the result: I accepted an offer from a different company that didn't require it. I doubt my life is that different than anyone else's, with a fu…

Yeah, we agree. We don't plan on asking everyone to do hours of work on their own. A lot of people don't have the time. However, we're also seeing a bunch of applicants who are good programmers, but become really stressed in interviews and do badly (they freeze). These people have trouble getting jobs. What we want to do is offer them the option of doing a larger project on their own, rather than our final interview.…

I think this project approach is excellent - this seems to map much better to real experience in a job.

Re: Three hundred programming interviews in thirty days

#43

Post by akg_67 »

eitally wrote: post #16
protonfish wrote: post #6

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of giving a candidate a project to do "in their own time." I recently had a interview that included this and I can share the result: I accepted an offer from a different company that didn't require it. I doubt my life is that different than anyone else's, with a fu…

My wife is in this position now, and it's not for a programming job. She works in pharmaceuticals, in pharmacovigilance & risk management. She does a lot of data analysis & writing (the latest report she wrote for Health Canada & the FDA was 461 pages). A company she's interviewing with asked her to do a sample writing task as part of her application/interview. She hasn't decided whether it's worth it or not. For som…

I work in data analysis and I decline any "test" requests for a job or project.

If you don't value my time before hiring, you definitely are not going to value my time after hiring. Also, if you are incapable of judging a fit based on resume, interviews, and just talking with me, you are not mature enough, experienced enough, smart enough and intelligent enough and have failed the test for being my potential coworker and boss.

Edit: The tests make perfect sense for entry level positions and entry level candidates but once someone requesting tests for experienced positions and candidates, it shows the immaturity of the company, group, team and people who are unable to assess suitability of candidate and place too much emphasis on technical skills and tests as a crutch. The best jobs in my career were those where coworkers and boss had good relationship dynamics. Technical excellence had nothing to do with how well we worked and delivered results.

Re: Three hundred programming interviews in thirty days

#44

Post by jbangert »

Very interesting methodology, but it would be very nice to correlate this data with long-term job performance. Interview decisions (of which of course you get more than long-term results, and they are clearer to quantify) are hopefully, but not necessarily an indicator of whether an employee works out for your company. Otherwise you run the risk of optimizing the quiz/screening process around metrics that influence your interview (i.e. test for how you personally weigh performance indicators, not for how these performance indicators actually affect performance)

Re: Three hundred programming interviews in thirty days

#45

Post by cnp »

ammon wrote: post #27
protonfish wrote: post #6

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of giving a candidate a project to do "in their own time." I recently had a interview that included this and I can share the result: I accepted an offer from a different company that didn't require it. I doubt my life is that different than anyone else's, with a fu…

Yeah, we agree. We don't plan on asking everyone to do hours of work on their own. A lot of people don't have the time. However, we're also seeing a bunch of applicants who are good programmers, but become really stressed in interviews and do badly (they freeze). These people have trouble getting jobs. What we want to do is offer them the option of doing a larger project on their own, rather than our final interview.…

I had an enormous amount of trouble interviewing / whiteboarding initially but I got used to it pretty fast. However, I would still prefer a take home test as I feel its a far better qualifier than an in-person interview.

Re: Three hundred programming interviews in thirty days

#46

Post by ammon »

rezistik wrote: post #33
ammon wrote: post #27

Earlier quoted context omitted.

Yeah, we agree. We don't plan on asking everyone to do hours of work on their own. A lot of people don't have the time. However, we're also seeing a bunch of applicants who are good programmers, but become really stressed in interviews and do badly (they freeze). These people have trouble getting jobs. What we want to do is offer them the option of doing a larger project on their own, rather than our final interview.…

I personally prefer it to a point. My only request when I get this type of interview is that I'm allowed to make the project and requirements public on my github for future interviewers. If I'm doing a free project I feel it's fair.

Well, that's hard. Standardizing evaluation (critical to reducing bias) requires that everyone (or a lot of people) see a similar problem. And when someone is working on their own time, I think it's important that there's not working code easily accessible online. Perhaps this can be overcome by asking candidates to talk through their code after they are done

Re: Three hundred programming interviews in thirty days

#47

Post by Buetol »

akg_67 wrote: post #43
eitally wrote: post #16

Earlier quoted context omitted.

My wife is in this position now, and it's not for a programming job. She works in pharmaceuticals, in pharmacovigilance & risk management. She does a lot of data analysis & writing (the latest report she wrote for Health Canada & the FDA was 461 pages). A company she's interviewing with asked her to do a sample writing task as part of her application/interview. She hasn't decided whether it's worth it or not. For som…

I work in data analysis and I decline any "test" requests for a job or project. If you don't value my time before hiring, you definitely are not going to value my time after hiring. Also, if you are incapable of judging a fit based on resume, interviews, and just talking with me, you are not mature enough, experienced enough, smart enough and intelligent enough and have failed the test for being my potential coworker…

A simple solution is for the company to offer you something for the time you spent doing the sample test (money, gift card, their product,...)

Re: Three hundred programming interviews in thirty days

#48

Post by Xyik »

donall wrote: post #13
michaelvkpdx wrote: post #5

Without any demographic data about who the interviewees were, this is basically useless. Are they all under 30, white and Asian men? Where?

I can see that that would be interesting information to have, but why is it absolutely necessary?

Fresh graduates are often better at programming / algorithm interviews than older experienced programmers who have been out of the interviewing game for a while. It's actually pretty important to know which demographic of users this data applies to.

Re: Three hundred programming interviews in thirty days

#49

Post by blue11 »

I found the article very interesting, but it seems to me that the metrics are the wrong ones. You cannot treat hire and no hire outcomes equally. Let's not forget that the goal of the interview process is to actually hire people. A successful hire is worth millions, a correct no-hire decision is just avoiding further losses of time and money. On the other hand, in terms of efficiency, a candidate that wasn't rejected early is the most expensive error. Well, not as expensive as actually hiring the wrong person, but it usually takes months to figure that out. Whereas the fact that a whole man-day was spent interviewing the wrong person on site becomes obvious in a few hours.

Here are the important metrics in my opinion, in order of decreasing importance:

- How many people were hired? Or what percentage of positions were filled? How long does it take to fill a position? Nothing in the article mentioned how many people were actually hired.

- False positives (people making it to the most expensive stage of the interview, typically a day-long on site interview, and being rejected there). What percentage of people that went to on site interviews got offers? Personally, I have always advocated processes that eliminate as many false positives as possible, even if it comes at the cost of some false negatives. Of course, you have to be careful not to filter out people too aggressively, because then you're just not going to hire anyone.

- False negatives (incorrectly rejecting good candidates early). By definition that's impossible to measure exactly. However, if you are not hiring fast enough, then maybe you have a problem with your screening process. At this point you could do an experiment and relax the screening process for half of the candidates and see what happens. But it could be just a sourcing problem, that is, you are not getting good candidates in the pipeline to begin with. It's very hard to tell whether you are being too harsh and not believing enough in people's abilities (or not willing to develop talent), or you are just not attractive to the kind of people that you want to hire.

Of course, all of the above is from the employer's point of view. If you are also trying to provide job seekers with a good service, then you can devise other metrics for success and for efficiency from their perspective.

(edited for formatting)

Re: Three hundred programming interviews in thirty days

#50

Post by dadrian »

>Suddenly, a significant percentage of the people who had spoken well about impressive-sounding projects failed, in some cases spectacularly, when given relatively simple programming tasks.

This indicates to me that either the "simple programming tasks" are not well-designed, or the the discussion about the past projects was not long enough. It still sounds like this interview process is only identifying candidates who are good at coding while someone is watching over their shoulder.

However, what I find to be the bigger issue with this article is that "success" is considered to be "passed the interview". Ultimately, all this article tells us are what currently correlates with qualities Triplebyte likes to see in candidates, not what correlates with good hires. To be fair, they do mention this at the end of the article.

Post reply on HN