Live data from Hacker News

We compiled a library of realistic engineering take-home tests and ranked them

trytapioca.com

181–190 of 190 posts

Re: We compiled a library of realistic engineering take-home tests and ranked them

#181
Disclaimer: one of my take home assignments I designed is part of the list compiled by the author.

---

Every interview process will inherently be a flawed one. Having hired several hundred engineers in my career, I believe that it is essential to try to create an interview environment where candidates feel they are given the best possible chance at showing their potential.

There's plenty of bias in engineering already that puts women, people of color and minorities at a disadvantage - and this shows through how most companies approach engineering hiring.

In one of my previous roles, we looked at who made it though to subsequent interview rounds and found that we favored candidates with very similar backgrounds to our own. More than we had thought, as it turns out, than we believed going into the analysis. Our solution to reducing this bias up front was to design take home assignments that tried to resemble the day to day job as much as possible, and with measurable criteria for what we wanted candidates to cover to determine their experience and skill.

It took a tremendous amount of work to get this process right, with a number of iterations to figure out the right balance for how much time a candidate should spend on an assignment and how much of day to day work our engineers should allocate to evaluating candidates.

Over the course of two years, we were able to achieve an almost equal split between women and men being hired into our engineering organization, less employee churn and a generally more vibrant workplace.

Take home assessments aren't a silver bullet, and they don't address every piece of the hiring puzzle. I do, however, believe that they are much better positioned to help interviewees perform at their best in an environment they feel comfortable in. This is especially true for people who today are less represented in software engineering roles.

At the end of the day, take home assignments are one part of a larger toolbox of tools companies must make use of to hire great people. Just like any other approach, take homes can be abused or misused, but I think they deserve more attention than most hiring managers give them.

Use them, or don't. Just make sure you hold yourself accountable to your own bias and try to reduce it so you don't miss out on great talent that don't fit your predefined view of what a great candidate looks like.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#182
post #120

Earlier quoted context omitted.

And I'd rather somebody admit that they took longer, say 4 hours to get to the 80% because they were a bit rusty, their IDE on their private machine wasn't as expected (e.g. Intellij CE vs the Ultimate at work), they had to look up how to bootstrap a service from 0 because they hadn't done that in ages even at work etc. than to under deliver a 20% solution in the 2 hours asked. Limits are fine if you ask me. Give me…

The limit is the limit. If you don't finish in the time given, I want to see what you did in that time.

I'm with the GP here, I'd much rather give them a few days and advise how much time they should spend on it than give them a strict timebox - it allows for a lot more flexibility with other responsibilities.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#183
post #66

Earlier quoted context omitted.

Is there literally any other field where there's an expectation to essentially do fairly intense studying to pass interviews that apparently have very little to do with day to day jobs? Maybe they exist I've never encountered them. The limit of my "studying" for an interview is to just learn a bit about the company, their strategy, etc. either by online research or by talking to people I know connected with them. ADD…

Is there literally any other field where there's an expectation that you will work for free 4 hours in order to be considered for the job?

Tons of them. How much time do you think the typical actor or dancer spends auditioning and preparing for the audition? How much time do you think people in many creative fields spend putting together portfolios? And if I'm going to interview with just about anyone, I'm going to spend at least 4 hours prepping for an interview.

I'm not a big fan of take homes in general but a time-consuming job search and interviews is absolutely normal beyond low-skill jobs in a time of labor shortages.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#184
post #163

Earlier quoted context omitted.

You don't want to ask questions?

Why would I want that?

Because it's an opportunity to learn more about the company beyond what's on their website? Honestly, if I interviewed someone and they had no interest in asking any questions, that's probably an easy reject.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#185

Earlier quoted context omitted.

While this certainly sounds like reasonable criteria for rating a take home, does this truly account for general candidate reluctance to do take homes due to the time commitment? Even 2-4 hours is more time than conventional Leetcode-style initial assessments, and many may rather use that time to interview with 2-4 companies instead of just one. I recently interviewed with Ramp and enjoyed their model of practical pr…

> Even 2-4 hours is more time than conventional Leetcode-style initial assessments, Hard disagree on this one. Unless your day-to-day work includes many leetcode style problems, you need to put in significant time training on leetcode if you want to pass the interview. You should be able to complete a take-home based on your current skill set. Yes, it may take 4-6 hours (or 8-10), and yes, that is a big ask of a cand…

The difference is that you are spending 4-10 hours of your time trying to pass the process for 1 company. If you instead invest 4-10 hours doing leetcode prep, you are studying to pass the process for LOTS of companies. Economies of scale is a huge factor here

Re: We compiled a library of realistic engineering take-home tests and ranked them

#186

Earlier quoted context omitted.

From the paper you link: "This article summarizes the practical and theoretical implications of 85 years of research in personnel selection. On the basis of meta-analytic findings, this article presents the validity of 19 selection procedures for predicting job performance and training performance and the validity of paired combinations of general mental ability (GMA) and the 18 other selection procedures. Overall, t…

Re: 3 - if you read the full paper you'll see that on its own a work sample test has (barely) the highest predictive value (but a lower confidence in that than GMA which is more heavily studied). This quote I think does more to demonstrate that GMA and integrity are less correlated than GMA and work product testing or structured interviews, which is intuitive. Re: 4 - I'm not sure why you would consider a take home t…

Yeah I think #1 and #2 are valid criticisms. My issue with the parent comment's line of reasoning is that it doesn't apply the same level of rigor to evaluating whiteboarding as it does to work sample tests. Our goal should be to identify which evaluation approach is most predictive based on the evidence available.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#187

Earlier quoted context omitted.

As a candidate, it's an awful feeling to spend time only to get ghosted. It's usually not malicious, just disorganization. From a hiring manager's perspective, it's too easy for things to slip through the cracks. When the volume of candidates increases, it's hard to keep track of all the zip file submissions + individual repos while making sure the eng team reviews them all. This is one of the problems my team is hop…

I didn't say or even imply that it was malicious. Regardless, it's disrespectful and unprofessional. I fail to see how adding yet another SaaS tool into the mix of the dozens that companies already use is somehow going to fix what is essentially a cultural problem. Applicant Tracking Systems have already been a thing for a long time now.

From talking to many hiring managers who use take-homes and using 3 ATSs myself, most engineering take-home processes don't run through the ATS.

I agree with you that it's disrespectful to candidates. My point is that it's not just a cultural problem - it's also a tooling problem.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#188

This is a phenomenal resource. I wish it also included the DevOps category -- k8s, aws, terraforms of the world.

You'll see DevOps as one of the options in the 'Roles' filter with 13 examples in the library.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#189

Hardware Reverse Engineer here. Would love to see some more tests geared towards my field. (Either reverse or hardware engineering)

I've never run into someone who that is their career - do you work for companies that are trying to take the competitions stuff apart, or legacy hardware, or is there some other stuff that's obvious that I'm missing?

Think more along the lines of "bad guys" vs "good guys" rather than corporate competition. Perfectly harmless systems are built everyday. But, the consumers of those systems may not always be harmless. Understanding - and eventually controlling - the inner workings of these custom systems [covertly in some circumstances] is often necessary to stop the bad guys.

Re: We compiled a library of realistic engineering take-home tests and ranked them

#190
post #86

Earlier quoted context omitted.

If you give every person on the street 2 hours of your time, you wouldn't be able to do your day job. I don't like take home tests either, but the point of them is removing the bad candidates from the pipeline so that people who interview them (often ICs, and these are some of the biggest expenses within the company) don't waste time on someone that is unlikely to be qualified. Personally when I'm interviewing and I…

If only there were this magical piece of paper that everyone presented when they applied listing all their relevant skills and experience so that the company can determine if it's worth giving them 2 hours of their time. All kidding aside, the efficient thing about resumes is that you write them once and use them everywhere. Coding samples are inefficient because you have to take 4h to write one for each company. Edi…

Right, and the issue with that is that plenty of people lie, and even more exxagerate what they did.
Post reply on HN