Live data from Hacker News

The business of takehome assessments

careerfair.io

51–60 of 62 posts

Re: The business of takehome assessments

#51

>I’m 27 and single. There lies the biggest advantage of a takehome assessment for me: I get to spend more time on it than someone with a wife and two kids. I noticed the same thing. It seems take homes are mainly geared towards pre-selecting young single people with a lot of free time to code on the side besides their main job and other responsibilities, or who have no other responsibilities. Now I'm also single and…

In my engineering days, I'd agree with this. IF we have spent a good amount of time talking and IF there is an affirmation of "we WILL review this code with you" (not just "if we like what we see, then we'll talk"), then I'd -consider- it.

If your idea of hiring is to spam me after I apply in Workday with a multi hour coding project to see if I merit the time speaking to even a recruiter, no, that job app is going in my circular file.

Re: The business of takehome assessments

#52

Earlier quoted context omitted.

How’s your career going? I’m often worried that I’ll look like a dummy when I have to look up basic loop syntax. I’m not dumb, and I’m somewhat experienced… I just write code in Python, Matlab, Fortran, Bash, whatever else. It is shocking that the basic shape of syntax can be so different! And also the degree to which it doesn’t actually matter. It’s just when starting a new file. My brain needs a little bit of synta…

My brain refuses to learn bash conditional syntax. Personally, when I’m about to write a conditional in bash I take it as a cue to switch to another language.

    for i in $( ls | grep “a complicated enough filter that I never have to use conditionals in my loop”) ;
    do
         echo -n “if I don’t want to work on ” $i 
        echo “ it is time to change languages” 
    done

Re: The business of takehome assessments

#53

After reaching my 10 yoe mark, I almost explicitly refuse take home tests. Yes if there was only one company I wanted to work with and I was interviewing with only them, I'll make an exception. Companies interview a lot of candidates to hire someone, as a result of which, you as a candidate have to interview 5-10 companies to get hired and get paid fairly, 15+ if you want to maximize your career + comp. There is no r…

If you spend 40 hours on this and it only gains you a 1% increase in salary then you earn that back in 2 years. I would say that it will generally earn you much more so personally I see it as a good investment. If you enjoy the take home you're also more likely to enjoy the role so it's also a filter for you. If you're not enjoying the take home then stop, but if you're finding it interesting then it's well worth it. These things are also for you to assess future Jon opportunities. The best take homes are as close to actual work as possible for a reason.

Re: The business of takehome assessments

#54

I don't think we'll ever solve the problem of interviewing software engineers. - Whiteboard coding tasks put a LOT of pressure on you to perform while being watched in a stressful high-stakes situation, which is a situation that the extreme majority of SWEs won't deal with, and many simply can't. They also tend to test for algorithms (inverting a binary tree, etc.), something that is unrelated to most actual jobs. -…

I feel like someone should cut a check after submitting someone to 5+ rounds. Or at least like a $200 Amazon gift card.

Re: The business of takehome assessments

#55
post #24

These are/were rampant in product design hiring. Especially "startups" or smaller companies. The direction is always the same "spend max a couple hours!" but the understanding is clearly you must spend significantly more. Some are cute with clearly non work related problem, like a previous company I worked for that did "design an app for a time machine" or similar. But many are very very obviously current problems th…

> The direction is always the same "spend max a couple hours!" but the understanding is clearly you must spend significantly more. I am in Product now, but when I was in SRE, I got one like this. "We don't want you to spend more than 3-4 hours on this", where "this" was: Build a log parser for streaming Apache CLF. The parser should: Keep a rolling monitor of the top 10 requests, displaying their velocity in req/sec…

I got a similar one once, like a fully working mocked banking system with currency exchange in 3 hours.

I just didn't proceed with the interview after that, if the staff can't manage to plan how long a test takes, there's no chance that they can plan anything accurately in the company either.

Re: The business of takehome assessments

#56

Those four companies he cites (Matter-App, Adcellerant, Symplicity, Bidsight) that tell you to spend x hours, then tell you "hey, but y'know, if you want to do [an open-ended number of additional hours for free], that'd be great!" should all go sit in the corner. Realistically, people are going to spend more time on their take-home assignments, because they perceive there's an advantage to doing so. It's not hard to…

This is what I don't care for, because it doesn't mirror the job at all. If it takes someone 20 hours to do a 3 hour project, that's a problem and it won't show up in the takehome.

It's almost the opposite of what the job actually entails. I rarely estimate that a project will take 3 hours and then I have 48 uninterrupted hours in which to accomplish it, no biggie if it takes more than 3. The job is typically the opposite, where I estimate it will take 3 hours and then I realized that I need to accomplish 40 hours of work in the 20 meeting-free work hours I have this week so I need to find a way to finish it in 90 minutes.

Re: The business of takehome assessments

#57
post #39
post #24

These are/were rampant in product design hiring. Especially "startups" or smaller companies. The direction is always the same "spend max a couple hours!" but the understanding is clearly you must spend significantly more. Some are cute with clearly non work related problem, like a previous company I worked for that did "design an app for a time machine" or similar. But many are very very obviously current problems th…

> The direction is always the same "spend max a couple hours!" Interestingly, I always took restrictions on time seriously. When I was told I should spend max 3 hours, I stopped latest at 3 hours, often earlier and discussed shortcommings of the solution based on spent time in the interview. That strategy is easier when you are not that interested in that particular job though.

Every job I tried this approach on were immediate rejections. Only solutions that completely fulfilled the requirements, with lots of testing, etc. were accepted for next steps. IE. I've never seen anyone who actually meant it when they gave a time limit.

Re: The business of takehome assessments

#58
post #10

The solution is simple: pay candidates for the estimated time the assignment should take, based on the median rate of the role they're applying for. Pay them regardless if they pass or fail the test. You should be able to avoid grifters looking for handouts if your screening process is any good. But I don't think this type of evaluation is a valuable signal anymore in the age of AI. Anyone could nowadays finish your…

I really like the PR-role-play interview, too. I do worry that it can devolve into the classic mind-reading interview—i.e., there is an extremely subtle bug (or perceived deficiency) in the code that a candidate is dinged for not seeing.

Re: The business of takehome assessments

#59
What I hate is that there's no often no discussion about your solution. I want to go "here, this is my solution", and then have a discussion with the people hiring where they might ask stuff like "you chose to do X this way, what was your reasoning?"

Also, enough with the fucking Roman numerals, Jesus Christ.

Re: The business of takehome assessments

#60
I recently designed a take-home. I made sure to pick a real, interesting, problem we faced, and one where there is a very high dynamic range of responses - it can distinguish between good and bad juniors (I know this because I let all my students try it if they want, which also gave me great deal flow of juniors without even needing to spend time on screening the bad ones), and it gives some information even at my level (I didn't arrive at the best possible solution myself, someone came up with a better solution to one subproblem).

I like respecting candidates' time, because I want the strongest candidates to not filter themselves out at the mention of a take-home. That's why I ask them to schedule 4 hours with me where at the start they will get the problem description and at the end they will submit what they have - this way they aren't pressured to put in more time because they know for sure that their competitors didn't - and I get to compare apples to apples.

I also designed the rules to minimize the perverse incentives set by the clock by giving multiple ways to get recognition for partial solutions.

This is the content of the document the candidate gets initially (they only get the actual problem later):

Meta

This is a real task we encountered and solved at Finubit. Out of respect for your time, we limit this task to 4 hours (of wallclock time, to remove the incentive to spend more than that to gain a competitive advantage). It took us much more than four hours to get the perfect solution, so don't worry, we don’t expect a full production-ready solution to 100% of the problem.

We ask that you approach this task as you would the first 4 hours out of as much time as you need to finish the task to a reasonable quality standard. We do want to see a working Proof Of Concept at the end of those 4 hours, like we would ask of any team member picking up a long, open-ended research task. The POC can be very limited in functionality, but functionality that it promises will be held to a standard of quality.

If you encounter ambiguities or mistakes, assume the most reasonable way to resolve them and make a note to check with Product that your understanding was correct. If you disagree with Product about something, make a note to push back against the design decision they made (but in the meantime assume the current spec, unless it's grossly wrong). If there's a subcomponent in your design that you don't know how to implement but believe to be solvable, it's OK to assume the existence of a black box that solves it.

Please take a moment to set an alarm clock. When the clock strikes 4 hours, please send us your code and research notes, and then go over the code and document everything you’d change before you consider the code production-ready and send us the annotated code.

We will be available to answer clarification questions if needed.

Then there is a checklist to reduce loss on technicalities (forgetting to set an alarm or send annotations).

(If you're in Israel, and you want to try it for fun, shoot me an email)

Post reply on HN