Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We compiled a library of realistic engineering take-home tests and ranked them

trytapioca.com

Post reply on HN
Print view View original

141–150 of 190 posts

Re: We compiled a library of realistic engineering take-home tests and ranked them

#141

Post by alexyang21 »

LewisJEllis wrote: post #58
twawaaay wrote: post #5

As an interviewer, I believe take home tasks are unfair and also poor return on investment. Take home tasks incentivise candidates to burn as much time as possible on the task. Consequently, they are a test of who is willing to put a week of work even when the instructions say it should not take more than 3 hours to complete. I prefer pair programming with the candidate on the premise that if I ask you to spend 2 hou…

Have you ever considered that solutions to your perceived problem might exist? I’ve done multiple different takehomes with some kind of time limit in place to avoid that sort of thing. Tools have this built in. Also, there are definitely candidates who would prefer a takehome. If you’re worried that a takehome might be unfair, you can offer folks a choice to either do the exercise on their own async, or to do it live…

I spoke to a head of engineering yesterday who asked candidates pick from 3 options:

1. Bring some work that you did previously (>4 hrs) and we'll discuss it.

2. Complete a 1-hour take-home.

3. Complete the 1-hour take-home live with one of our engineers.

He said that 98% of candidates chose option #2. I was surprised at how high it was!

Re: We compiled a library of realistic engineering take-home tests and ranked them

#142

Post by alexyang21 »

TylerE wrote: post #63
NeverFade wrote: post #47

Earlier quoted context omitted.

How do you know they clearly took over two hours? We love talking about productivity here. You're telling me employers will systematically fail the candidates who performed most productively in their process, because of a speculation they took too much time?

Well, it's pretty simple if you don't given them the task until they agree they're ready to start, and then record when you receive the submission. Any candidate capable of actually inventing a time machine should certainly be hired.

Haha yeah we implement 'soft time-boxing' by tracking git commit timestamps. It's not as stressful as having a visible timer and we won't block a submission that exceeds the recommended time, but reviewers can clearly see who took longer and their last commit within the suggested time, which helps to create an apples-to-apples comparison.

No candidate wants to be entered in the Hunger Games for who has the most time to sink into a take-home.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#143

Post by alexyang21 »

skhameneh wrote: post #90
twawaaay wrote: post #5

As an interviewer, I believe take home tasks are unfair and also poor return on investment. Take home tasks incentivise candidates to burn as much time as possible on the task. Consequently, they are a test of who is willing to put a week of work even when the instructions say it should not take more than 3 hours to complete. I prefer pair programming with the candidate on the premise that if I ask you to spend 2 hou…

After doing a few take home assignments, I've learned many won't give the courtesy of Q&A afterwards; many won't give any feedback by default. I loved giving live code sessions, the more open-ended the better. Give an objective, explain there's no wrong answers, answer all questions, give answers when asked, keep it simple, and gauge response. I found it exposed attitude and knowledge.

I blame the legal teams at large companies who are worried that a well-meaning hiring manager will give feedback that could be used in a lawsuit.

Love that your approach focuses on understanding thought process! IMO many companies focus too much on raw technical skills, when softer skills like attitude may be more predictive of on-the-job performance. My team is working to elicit the same signal in a take-home format (since it's more scalable), but I think the best is a combination of the two: short (~1 hr) take-home + follow-up live session on the work that was already started.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#144

Post by morelisp »

irf1 wrote: post #124
morelisp wrote: post #112

Earlier quoted context omitted.

I mean you do you, but it's a total farce to call this paying people for their time. You're paying people who win your little contest, which is what hiring already is before your startup tries to insert itself as a middleman.

I'd call it an early experiment and what we could afford at the time to evaluate candidates for a SWE summer internship. Maybe other companies will like this approach and offer challenges that pay much better, with different setups & time horizons. Ex a user turnt his challenge into a 3-month contract-to-hire arrangement that paid $15k to the person who did the work & got the job. Ultimately it's about getting the ri…

> what we could afford at the time to evaluate candidates for a SWE summer internship

You didn't even hire them as FT? Christ, what a submarine charade this entire thread has been.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#145

Post by dymk »

lost953 wrote: post #35
ryandrake wrote: post #11

Earlier quoted context omitted.

If you are interviewing for 10-20 jobs in a particular week, a 2-4 hour take home test is a full time job 40 hours worth of work. I would accept a 2-4 hour time commitment but only much later in the interview process, when much of the interviewing was done and the list was whittled down to 2-3 serious companies.

Seriously, if a company wants me to spend more than an hour of my time on something for them, they better be including a check as well.

Less competition for everybody else looking at the position, then.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#146

Post by TylerE »

stephenhogsten wrote: post #106
ghaff wrote: post #56

Earlier quoted context omitted.

And a reasonable response in any case is "Yah, I know you said that but I don't like to do sloppy work and there were clearly a number of different edge cases that needed to be handled for this to work reliably."

I've seen a middle ground where a timebox is enforced, but there's a discussion question for what next steps would be

Yes, this is what we do at my company. It's the only way this sort of thing is valid IMO... it's not a race, but I want to see demonstration of reasonable output.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#147

Post by jameshart »

zaphar wrote: post #21
twawaaay wrote: post #5

As an interviewer, I believe take home tasks are unfair and also poor return on investment. Take home tasks incentivise candidates to burn as much time as possible on the task. Consequently, they are a test of who is willing to put a week of work even when the instructions say it should not take more than 3 hours to complete. I prefer pair programming with the candidate on the premise that if I ask you to spend 2 hou…

Counter-Point. If the assignment is properly sized: no more than an hour. Then the candidate doing it at home using their system and their tools gives you a more accurate picture of their actual capabilities than a whiteboard or doing it on your hardware in an unfamiliar setting with a stranger giving every appearance of critiquing your work. This is an interview after all. The power balance whether intended or not i…

> the candidate doing it at home using their system and their tools gives you a more accurate picture of their actual capabilities

We’re just gonna run with the assumption that anyone who’s applying for a job has their own development machine.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#148

Post by alexyang21 »

time_to_smile wrote: post #62
_odey wrote: post #15

Now take that list, throw it in the rubbish bin and wash your hands thoroughly. Take home challenges are simply an absurd waste of time and a huge disrespect towards your candidates personal time. You ask them to spend 4 or more hour of their free time on a useless, throwaway excersise that you "review" in 5 minutes and you conclude they're good enough because they laid out code the way you like it... You never see t…

Another huge problem with take home challenges is that they are one time use. At least if you're grinding leet code questions you're spending time that is useful at multiple companies. I have a big folder on my computer of homeworks I've finished for various companies over the years. If I recall I passed all of them, it's sort of annoying that I can't simply hand over this list of other projects I've passed and say "…

Did the companies tell you no when you asked them if you could share a previous project instead of completing theirs? From talking to many hiring managers, they also don't want to waste your time, and most will be happy to accept an alternative if it show similar skills to the ones they're looking for. I'm even seeing more teams offering this as an explicit option now.

There will always be a few who say no (IMO this could be a warning sign), but it doesn't hurt to ask :)

Re: We compiled a library of realistic engineering take-home tests and ranked them

#150

Post by anchochilis »

twawaaay wrote: post #139
anchochilis wrote: post #76

Earlier quoted context omitted.

As a candidate, I vastly prefer a take-home to live whiteboarding or pairing. That background sensation of being observed and evaluated greatly interferes with my ability to concentrate and perform at my best. (I'll note that I have no problem pairing in real-world work scenarios! I love discussing problems with my team. It's the artificial pressure injected by the interview process that really gets to me)

It is not about what candidates prefer. If that was the case, there would be no whiteboards or any kind of exercises at all.

It actually is, because the interview process influences the size, diversity, and caliber of the pool of candidates who apply to your organization, complete the interview loop and accept offers.

If you have an unlimited engineering budget or a big, attractive name then by all means, force candidates to jump through as many hoops as you'd like, you'll still be able to fill open reqs. But the majority of companies are not in that position and in fact struggle to attract and retain technical talent.

Post reply on HN