We compiled a library of realistic engineering take-home tests and ranked them

Studies [1] show that a work sample test is the best predictor of candidate performance on the job, which is why many software engineering teams use take-home tests as one step in their hiring process. But designing an effective test is difficult and time-consuming. For example, candidates are reluctant to complete tests that are too long or not engaging enough. But make them too short and teams won’t get the signal they need for a proper evaluation.

To encourage more thoughtful test design (and hopefully save future candidates from the worst offenders), my team compiled the largest library of non-“whiteboard” take-home tests that real engineering teams have used. You’ll find the challenges that Stripe and Microsoft gave to their full-stack candidates, front-end tests from Tailwind and Rivian, and back-end ones from Basecamp and Revolut. Whether you’re looking to evaluate an Android, DevOps, or Data Science candidate, a bootcamp grad, or senior engineer, we found a few options for each.

Having built 20+ tests ourselves, we also rated the design of each test. The criteria for a 5-star rating:

1. Tests for skills highly relevant to those required for the position

2. Includes a well-written description of the prompt and even motivation for using a take-home test

3. Sets clear expectations for candidates (e.g. time requirements, evaluation criteria, submission details)

4. Asks for a reasonable time commitment from candidates (A few notes: - We found most of these test prompts in public GitHub repos, usually owned by the hiring team but occasionally in the candidate-owned submission. We sifted through hundreds of tests and filtered out those overly focused on algorithms (aka LeetCode), leaving us with 142 tests in the library.

- The larger and more recognizable companies didn’t always have the best tests. Some of the most interesting prompts we found were from smaller teams (e.g. YC startups). This shouldn’t be surprising. Startups need to design candidate-friendly hiring experiences to compete for talent against more established players.

- There were common themes among the tests we found. For example, front-end candidates were often given a Figma design + content feed to implement, while back-end candidates had to implement an API given a set of requirements. Data scientists were usually given a data set to clean, analyze, and submit a Jupyter notebook with their findings.

- We’ll continue to update this library and add descriptions of each test so it’s easier to compare.

Have feedback, or another take-home test we should add? We’d love to hear from you!

[1] The Validity and Utility of Selection Methods in Personnel Psychology (https://www.researchgate.net/publication/232564809_The_Valid...)

Very useful, thanks!

While this certainly sounds like reasonable criteria for rating a take home, does this truly account for general candidate reluctance to do take homes due to the time commitment? Even 2-4 hours is more time than conventional Leetcode-style initial assessments, and many may rather use that time to interview with 2-4 companies instead of just one.

I recently interviewed with Ramp and enjoyed their model of practical problems, like a take home, but condensed into just one hour with less of the end-to-end expectations of a true take home.

After scanning the first page, I quickly noticed "Want something unique to your team? Tell us what you need."

I'm not a marketing guy (by any stretch) so maybe my opinion isn't worth much, but I think this is outstanding - provide something already useful, and then invite people to purchase your customized even more useful stuff.

It's always nice to see something well done — kudos!

(You seem to be focused on software development, but if you have anything along the lines of practical math used in a lab/shop/factory environment around machining, mixing, monitoring, testing, etc., It'd be great to hear about it)

Or it'll just cause the leetcode bar to be moved to fit the tests with "stronger signals".

Goodhart's Law will basically apply to any exam under capitalism: "When a measure becomes a target, it ceases to be a good measure."

If these jobs didn't offer more compensation and prestige than other positions, matching candidate competency to job requirements would be much more achievable, but as it is now, it's an adverseal system that demands gamification (because when any candidate plays the game, if you don't, you are actively harming your material conditions with a lower position and smaller salary)