We compiled a library of realistic engineering take-home tests and ranked them • Hacker News Board

Re: We compiled a library of realistic engineering take-home tests and ranked them

#11

Post by ryandrake » Thu, Aug 25, 2022, 6:45 PM UTC

noisenotsignal wrote: post #6

alexyang21 wrote: post #2

Studies [1] show that a work sample test is the best predictor of candidate performance on the job, which is why many software engineering teams use take-home tests as one step in their hiring process. But designing an effective test is difficult and time-consuming. For example, candidates are reluctant to complete tests that are too long or not engaging enough. But make them too short and teams won’t get the signal…

While this certainly sounds like reasonable criteria for rating a take home, does this truly account for general candidate reluctance to do take homes due to the time commitment? Even 2-4 hours is more time than conventional Leetcode-style initial assessments, and many may rather use that time to interview with 2-4 companies instead of just one. I recently interviewed with Ramp and enjoyed their model of practical pr…

If you are interviewing for 10-20 jobs in a particular week, a 2-4 hour take home test is a full time job 40 hours worth of work.

I would accept a 2-4 hour time commitment but only much later in the interview process, when much of the interviewing was done and the list was whittled down to 2-3 serious companies.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#12

Post by endisneigh » Thu, Aug 25, 2022, 6:46 PM UTC

I actually don’t mind take home tests, but I want the company to also burn resources as to disincentivize ghosting.

Problem with take homes is that you can spend hours and then they can spend seconds or minutes reviewing.

At least with traditional interviews the time spent is equal for both parties.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#13

Post by stephenhogsten » Thu, Aug 25, 2022, 6:46 PM UTC

twawaaay wrote: post #5

As an interviewer, I believe take home tasks are unfair and also poor return on investment. Take home tasks incentivise candidates to burn as much time as possible on the task. Consequently, they are a test of who is willing to put a week of work even when the instructions say it should not take more than 3 hours to complete. I prefer pair programming with the candidate on the premise that if I ask you to spend 2 hou…

These are great points.

How do you feel about take-homes that have an enforced cap at 60-120 minutes to try to remove the competition on time investment?

Do you learn more than it's possible to learn asynchronously? Most take-homes only have candidates write code, but it's possible to understand a candidate more deeply by asking questions about, for example, how they approached the decisions they made.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#14

Post by lelandfe » Thu, Aug 25, 2022, 6:47 PM UTC

As there are no tests rated 1 star, I wonder if the scoring is too lax.

I've taken some of these – shocked that Algolia's made it to a 3 star. Look at how much work this is: https://github.com/algolia/solutions-hiring-assignment

Re: We compiled a library of realistic engineering take-home tests and ranked them

#15

Post by _odey » Thu, Aug 25, 2022, 6:48 PM UTC

Now take that list, throw it in the rubbish bin and wash your hands thoroughly.

Take home challenges are simply an absurd waste of time and a huge disrespect towards your candidates personal time. You ask them to spend 4 or more hour of their free time on a useless, throwaway excersise that you "review" in 5 minutes and you conclude they're good enough because they laid out code the way you like it... You never see the really important things that you should be screening for: how they explain their approach, their though process, how well they communicate, the compromises they had to make to fit within the timeframe, and why they'd made those compromises, how they react and adapt to changing requirements and feedback...

It's just flawed on so many levels...

As a new Engineering Manager I've eliminated the take home challenge in my new company in favour of this process:

- You have open source code or closed code that you can show? Walk me through it. Our best engineers passed the interview this way.

- You don't have code to show, 1 hour live coding challenge, focus on communication, search want you want online, ask us any questions, and explain what you're doing.

We had about 40 such interviewes this year, one person refused the live challenge, 11 passed this stage, and we ended up with 7 hires accepting the offer.

If you lack inspiration, use this: https://github.com/guardian/coding-exercises

Re: We compiled a library of realistic engineering take-home tests and ranked them

#16

Post by llimllib » Thu, Aug 25, 2022, 6:49 PM UTC

All of ours[1] are given three stars but I would love to know why! I feel like they meet your criteria, but who knows.

Would love yours, or anybody else's feedback on our problems. We do care about them, and wrote about why we use homework here: https://adhoc.team/2018/02/26/why-we-use-homework-to-recruit...

[1]: https://homework.adhoc.team/

Re: We compiled a library of realistic engineering take-home tests and ranked them

#17

Post by kevmo314 » Thu, Aug 25, 2022, 6:49 PM UTC

stephenhogsten wrote: post #13

twawaaay wrote: post #5

As an interviewer, I believe take home tasks are unfair and also poor return on investment. Take home tasks incentivise candidates to burn as much time as possible on the task. Consequently, they are a test of who is willing to put a week of work even when the instructions say it should not take more than 3 hours to complete. I prefer pair programming with the candidate on the premise that if I ask you to spend 2 hou…

These are great points. How do you feel about take-homes that have an enforced cap at 60-120 minutes to try to remove the competition on time investment? Do you learn more than it's possible to learn asynchronously? Most take-homes only have candidates write code, but it's possible to understand a candidate more deeply by asking questions about, for example, how they approached the decisions they made.

How would such a cap be enforced? If it's due 120 minutes from the time it's given, it's no longer a take home interview, it's a remote interview.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#18

Post by xtracto » Thu, Aug 25, 2022, 6:49 PM UTC

twawaaay wrote: post #5

As an interviewer, I believe take home tasks are unfair and also poor return on investment. Take home tasks incentivise candidates to burn as much time as possible on the task. Consequently, they are a test of who is willing to put a week of work even when the instructions say it should not take more than 3 hours to complete. I prefer pair programming with the candidate on the premise that if I ask you to spend 2 hou…

As a hiring manager I wholwheartedly agree with this. Take home tests introduce so many uncontrolled variables, I wouldn't be able to measure and compare two candidates.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#19

Post by smegsicle » Thu, Aug 25, 2022, 6:51 PM UTC

kevmo314 wrote: post #17

stephenhogsten wrote: post #13

Earlier quoted context omitted.

These are great points. How do you feel about take-homes that have an enforced cap at 60-120 minutes to try to remove the competition on time investment? Do you learn more than it's possible to learn asynchronously? Most take-homes only have candidates write code, but it's possible to understand a candidate more deeply by asking questions about, for example, how they approached the decisions they made.

How would such a cap be enforced? If it's due 120 minutes from the time it's given, it's no longer a take home interview, it's a remote interview.

and however it was enforced, it would increase the stress a great deal

Re: We compiled a library of realistic engineering take-home tests and ranked them

#20

Post by convolvatron » Thu, Aug 25, 2022, 6:52 PM UTC

pmonza wrote: post #3

It's painful to imagine how many teams miss out on candidates because their take-home or technical interviews are not extracting the right signals. Having access to a library like this will save so much time on getting new ideas and iterating. At the very least, I hope it saves the effort of running into useless leetcode libraries.

too many companies are using the take home as an initial filter. twice now I've spent a couple days really polishing the assignment only to discover later that there is absolutely no chance we will work together well.

at least make me _want_ the job before you start placing demands on my time and those of who I might provide as references.