Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Three hundred programming interviews in thirty days

blog.triplebyte.com

Post reply on HN
Print view View original

61–70 of 248 posts

Re: Three hundred programming interviews in thirty days

#61

Post by ammon »

techiferous wrote: post #52
protonfish wrote: post #6

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of giving a candidate a project to do "in their own time." I recently had a interview that included this and I can share the result: I accepted an offer from a different company that didn't require it. I doubt my life is that different than anyone else's, with a fu…

I've had a different experience on the hiring side of the table. The coding exercise that we provide as part of the interview has been very successful in surfacing the skill and talent of the programmer. By using that screening device, we have assembled a team of very sharp programmers. It's totally worth filtering out people who choose not to spend the time. Even though we end up losing some potential candidates, it…

The problem is that you don't actually have a clear measure of who you're rejecting. Every company uses a similar argument to support their (very different in every case) interview process. The fact that it allowed you to build a great team does not mean that you could not have built a better team by giving other options as well.

Re: Three hundred programming interviews in thirty days

#62

Post by panorama »

Naomarik wrote: post #29

I got a phone interview with Harj and wasn't considered for anything further. I'm not sure what kind of hackers they were looking for, but I've been directly involved with creating the infrastructure used in marketing campaigns with the likes of CNN, McDonalds, Infiniti, and more. I've turned an idea into a company with 8 full time employees and have investors seriously interested in one of my side projects. I'm curr…

I fell through at that stage as well, which I think was surprising only because their value prop is connecting you to YC startups. I'm a former early employee at a YC startup having contributed to nearly half of its codebase and other YC startups constantly attempt to recruit me.

Oh well, I'm at a non-YC startup now and very happy. I guess the discrepancy lies with Triplebyte selecting engineers that Triplebyte wants, not what is actually representative of what YC startups want. Overall it's still probably a decent signal—you'd have to expect some false negatives here and there (and I'm not a rockstar dev either).

Re: Three hundred programming interviews in thirty days

#63

Post by gwbas1c »

Many fields require serious professional certification. You can't become a doctor unless you go through a board certification that includes simulated patient interaction. Likewise, you can't build a bridge or a dam without engineering certification.

IMO, this article demonstrates the need to certify software engineers; using a process similar to the interviewing process described. Therefore, when hiring, we can skip most of the "do you know how to code" and get down to cultural fit and mutual interest.

Re: Three hundred programming interviews in thirty days

#64

Post by technofiend »

eitally wrote: post #16
protonfish wrote: post #6

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of giving a candidate a project to do "in their own time." I recently had a interview that included this and I can share the result: I accepted an offer from a different company that didn't require it. I doubt my life is that different than anyone else's, with a fu…

My wife is in this position now, and it's not for a programming job. She works in pharmaceuticals, in pharmacovigilance & risk management. She does a lot of data analysis & writing (the latest report she wrote for Health Canada & the FDA was 461 pages). A company she's interviewing with asked her to do a sample writing task as part of her application/interview. She hasn't decided whether it's worth it or not. For som…

I had a similar experience last year applying for a major technology company. Although I still lightly code in the devops sense (process automation) and to do typical management data analysis my day job is managing people, processes and crises.

If you have billion dollar trading systems and need a cool-headed guy who can talk to VPs or directors in one breath and SAs, DBAs, SAN, Network, etc guys in the next I'm your man.

I applied to head a global support organization, with teams in (from memory) North America, India and Asia Pacific. I'm pretty sure day-to-day would be a mix of employee career management, forward planning with my application development partners to make sure we were prepared capacity and training-wise to take on whatever's next, and crisis management of production issues and the like. Business as usual as they say.

Yet I found myself answering computational complexity questions about hash map insertions. To be fair, before I became quite so management-centric I've found plenty of badly performing code via the usual tools (dtrace, strace, thread dumps), and sql queries using analogous database tools, but at some point you have to rely on technical people to do their jobs.

Am I wrong to think if I'm going to manage a 60-100 person organization across three timezones I don't really need to be under the hood in the code any longer? Or am I clueless?

Re: Three hundred programming interviews in thirty days

#65

Post by odonnellryan »

protonfish wrote: post #6

It's great to see some objective research being done on this, and I am very interested in following the results. They mention evaluating the effectiveness of giving a candidate a project to do "in their own time." I recently had a interview that included this and I can share the result: I accepted an offer from a different company that didn't require it. I doubt my life is that different than anyone else's, with a fu…

I think it depends on the project. I'm looking to switch roles, and some projects are very small: solve a simple algo question. Should take less than an hour, which is less than the average interview, anyway! So no problem.

Other projects are enormous. 40-80 HRs. of work, easily (write an entire web-app!) I did one of these... did not get the offer. I would not want to complete another.

Re: Three hundred programming interviews in thirty days

#66

Post by brudgers »

Buetol wrote: post #47
akg_67 wrote: post #43

Earlier quoted context omitted.

I work in data analysis and I decline any "test" requests for a job or project. If you don't value my time before hiring, you definitely are not going to value my time after hiring. Also, if you are incapable of judging a fit based on resume, interviews, and just talking with me, you are not mature enough, experienced enough, smart enough and intelligent enough and have failed the test for being my potential coworker…

A simple solution is for the company to offer you something for the time you spent doing the sample test (money, gift card, their product,...)

How about a car? Then it is obviously worth the applicant's time and since a professional can deliver or destroy a Bimmer's worth of corporate value on any given decision it easily pays for itself in identifying good candidates and weeding out bad ones.

Re: Three hundred programming interviews in thirty days

#68

Post by ammon »

KingMob wrote: post #55
ammon wrote: post #15

Earlier quoted context omitted.

Hey! Author here. Yes, we did not know that the screening steps were meaningful, so for the first 300 applicants, we interviewed everyone, even people who performed badly. We then looked for correlation between screening step scores, and programming interview results. Doing well on the fizzbuzz problems was not very correlated. By dropoff I mean people who left during a step, and never logged back into our site.

I appreciate that you guys might not be statisticians, but if you're going to try and analyze data like this, you simply must address survival bias. As it stands, these data are meaningless unless you assume dropouts are completely unrelated to your screening. You claim doing well on the Fizzbuzz wasn't correlated with interview performance, but you also said "We saw twice the drop off rate on the coding problems as…

All candidates did both screens (quiz and fizzbuzz). The correlations were calculated against the same population. Now, I agree that survivor bias could affect the quality of these results (we know nothing about the significant % of people who dropped out). But it's not really possible to solve that problem outside of a lab. I don't think it's an argument to not do analysis. For now we're simply trying to minimize the dropoff rate, and maximize correlation. The quiz was better at both.

Re: Three hundred programming interviews in thirty days

#69

Post by hayksaakian »

ammon wrote: post #46
rezistik wrote: post #33

Earlier quoted context omitted.

I personally prefer it to a point. My only request when I get this type of interview is that I'm allowed to make the project and requirements public on my github for future interviewers. If I'm doing a free project I feel it's fair.

Well, that's hard. Standardizing evaluation (critical to reducing bias) requires that everyone (or a lot of people) see a similar problem. And when someone is working on their own time, I think it's important that there's not working code easily accessible online. Perhaps this can be overcome by asking candidates to talk through their code after they are done

if you change the assessment every so often, you could permit people to release their work eventually.

Re: Three hundred programming interviews in thirty days

#70

Post by ammon »

jbangert wrote: post #44

Very interesting methodology, but it would be very nice to correlate this data with long-term job performance. Interview decisions (of which of course you get more than long-term results, and they are clearer to quantify) are hopefully, but not necessarily an indicator of whether an employee works out for your company. Otherwise you run the risk of optimizing the quiz/screening process around metrics that influence y…

I 100% plan to do that. It's a very exciting idea
Post reply on HN