Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We compiled a library of realistic engineering take-home tests and ranked them

trytapioca.com

Post reply on HN
Print view View original

21–30 of 190 posts

Re: We compiled a library of realistic engineering take-home tests and ranked them

#21

Post by zaphar »

twawaaay wrote: post #5

As an interviewer, I believe take home tasks are unfair and also poor return on investment. Take home tasks incentivise candidates to burn as much time as possible on the task. Consequently, they are a test of who is willing to put a week of work even when the instructions say it should not take more than 3 hours to complete. I prefer pair programming with the candidate on the premise that if I ask you to spend 2 hou…

Counter-Point. If the assignment is properly sized: no more than an hour. Then the candidate doing it at home using their system and their tools gives you a more accurate picture of their actual capabilities than a whiteboard or doing it on your hardware in an unfamiliar setting with a stranger giving every appearance of critiquing your work. This is an interview after all. The power balance whether intended or not is very much different than an actual peer review

If a candidate chooses to burn a whole week working on it that can be safely left up to them I think. But a take home test is not unfair because they decided to burn that much time on it. It is an attempt to let them give the clearest most accurate signal of their skill level and approach to problems. I would argue that not even pair programming accomplishes that as well as a take home assignment.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#22

Post by inanutshellus »

kevmo314 wrote: post #17
stephenhogsten wrote: post #13

Earlier quoted context omitted.

These are great points. How do you feel about take-homes that have an enforced cap at 60-120 minutes to try to remove the competition on time investment? Do you learn more than it's possible to learn asynchronously? Most take-homes only have candidates write code, but it's possible to understand a candidate more deeply by asking questions about, for example, how they approached the decisions they made.

How would such a cap be enforced? If it's due 120 minutes from the time it's given, it's no longer a take home interview, it's a remote interview.

Presumably the difference is that the interviewee gets to choose when to start the clock.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#23

Post by coxmichael »

twawaaay wrote: post #5

As an interviewer, I believe take home tasks are unfair and also poor return on investment. Take home tasks incentivise candidates to burn as much time as possible on the task. Consequently, they are a test of who is willing to put a week of work even when the instructions say it should not take more than 3 hours to complete. I prefer pair programming with the candidate on the premise that if I ask you to spend 2 hou…

How do you structure the 2 hour session?

Would be interesting to hear which kinds of tasks you use in a short space of time.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#24

Post by stephenhogsten »

smegsicle wrote: post #19
kevmo314 wrote: post #17

Earlier quoted context omitted.

How would such a cap be enforced? If it's due 120 minutes from the time it's given, it's no longer a take home interview, it's a remote interview.

and however it was enforced, it would increase the stress a great deal

True. More stress than an untimed take-home, but I would think less stress than a normal interview (for most engineers, at least).

Re: We compiled a library of realistic engineering take-home tests and ranked them

#25

Post by alexyang21 »

lelandfe wrote: post #14

As there are no tests rated 1 star, I wonder if the scoring is too lax. I've taken some of these – shocked that Algolia's made it to a 3 star. Look at how much work this is: https://github.com/algolia/solutions-hiring-assignment

Yeah I agree that a test that's super long is awful for candidates. We did our best to be objective in grading, so we didn't want to dock more stars based on our subjective opinion on what's considered reasonable.

FWIW my team believes that test designers should strive to scope tests to no more than 1-2 hours (ideally 1).

The reason why there aren't any 1-star tests isn't because they don't exist, but because we didn't think anyone would want to see them. Cataloguing these was a ton of work (we sifted through hundreds of tests). Including the 1-star ones seemed like it would only shame companies who used them.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#27

Post by stephenhogsten »

inanutshellus wrote: post #22
kevmo314 wrote: post #17

Earlier quoted context omitted.

How would such a cap be enforced? If it's due 120 minutes from the time it's given, it's no longer a take home interview, it's a remote interview.

Presumably the difference is that the interviewee gets to choose when to start the clock.

That's what I had in mind. You would need a tool to enforce it.

At that point "take-home" may not be the best name anymore, but "remote interview" just sounds like it's an interview over video. Maybe "async interview"?

Re: We compiled a library of realistic engineering take-home tests and ranked them

#28

Post by blueberrychpstx »

twawaaay wrote: post #5

As an interviewer, I believe take home tasks are unfair and also poor return on investment. Take home tasks incentivise candidates to burn as much time as possible on the task. Consequently, they are a test of who is willing to put a week of work even when the instructions say it should not take more than 3 hours to complete. I prefer pair programming with the candidate on the premise that if I ask you to spend 2 hou…

Best of both worlds to use this compilation as inspiration for one’s own company pie programming sessions.

I don’t find it useful to evaluate whether or not a potential hire can map and reduce a list in JavaScript or fix some silly react bug thags so obvious it hurts. Seeing how they approach real problems, architecting solutions on the fly etc are things I enjoy doing but I’m not as much on the hiring side so YMMV

Re: We compiled a library of realistic engineering take-home tests and ranked them

#29

Post by mellavora »

noisenotsignal wrote: post #6
alexyang21 wrote: post #2

Studies [1] show that a work sample test is the best predictor of candidate performance on the job, which is why many software engineering teams use take-home tests as one step in their hiring process. But designing an effective test is difficult and time-consuming. For example, candidates are reluctant to complete tests that are too long or not engaging enough. But make them too short and teams won’t get the signal…

While this certainly sounds like reasonable criteria for rating a take home, does this truly account for general candidate reluctance to do take homes due to the time commitment? Even 2-4 hours is more time than conventional Leetcode-style initial assessments, and many may rather use that time to interview with 2-4 companies instead of just one. I recently interviewed with Ramp and enjoyed their model of practical pr…

> Even 2-4 hours is more time than conventional Leetcode-style initial assessments,

Hard disagree on this one. Unless your day-to-day work includes many leetcode style problems, you need to put in significant time training on leetcode if you want to pass the interview.

You should be able to complete a take-home based on your current skill set. Yes, it may take 4-6 hours (or 8-10), and yes, that is a big ask of a candidate, but leetcode can take 5 to 10x prep time and you still can muff the interview.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#30

Post by ghaff »

kevmo314 wrote: post #17
stephenhogsten wrote: post #13

Earlier quoted context omitted.

These are great points. How do you feel about take-homes that have an enforced cap at 60-120 minutes to try to remove the competition on time investment? Do you learn more than it's possible to learn asynchronously? Most take-homes only have candidates write code, but it's possible to understand a candidate more deeply by asking questions about, for example, how they approached the decisions they made.

How would such a cap be enforced? If it's due 120 minutes from the time it's given, it's no longer a take home interview, it's a remote interview.

It doesn't need to be proctored. It would be easy to start a remote clock and the solution needs to be in by x minutes after the start. It can even be pretty generous. The idea is presumably not to see how fast they can sprint but just put some time-boxing in place so some people aren't taking days.

People can get help of course but that's going to be the case absent effectively a remote proctored assignment. And now you're back to effectively in-person.

Post reply on HN