What are some principles in test design that make it revealing and easy to time-box, regardless of role/stack? Any specific test you designed or solved as an interviewee that stands out?
Ask HN: How to design a good technical interview test (take home or live)
1–10 of 21 posts
Re: Ask HN: How to design a good technical interview test (take home or live)
#2That sounds like you want to keep people away from your organization. Why not ask them these questions during the interview? Usually ask questions around the issues that your organization is facing. See how they would solve that problem. Often times, teams get intimidated by strong talent so they often pass on these scouts, but your team should be the one that is providing you with the problems for these candidates to solve their riddles =p
Always present real time challenges, so that you are helping the teams pass obstacles and not providing tests that make no sense in teams therefore resulting in steep learning curves.
Re: Ask HN: How to design a good technical interview test (take home or live)
#31. Focus on a simplified version of a real problem you had to solve at your current job (MIN(arbitrary))
2. Needs to be explainable in 1 page with example data (MIN(completely))
3. Needs to not require any complex interaction. No web request, no complex libraries, just a unit testable piece of code. (MAX(impl speed))
4. Make it fun and something you and all your coworkers could easily do.
This has been my approach and it's been a good filter. If it's done before the interview have them come in and add a feature. Make sure they bring a machine with their prefered IDE / editor to maximize similarity to their real programming flow. Offer to do something else while they code so they don't feel "watched"
Re: Ask HN: How to design a good technical interview test (take home or live)
#4None of this at all. People already take time for multiple interview stages. Ask targeted questions, but don't trouble them with more 'work'. Ask for references.
Re: Ask HN: How to design a good technical interview test (take home or live)
#5I encourage you to see what happens when you leave testing out of the interview.
Learn about them, ask them to participate in a code review if you have to, pretend you work with them already.
Re: Ask HN: How to design a good technical interview test (take home or live)
#6no take homes pls
Re: Ask HN: How to design a good technical interview test (take home or live)
#7For jobs that I am excited about I wouldn't mind doing a take home test as long as the project was interesting.
Re: Ask HN: How to design a good technical interview test (take home or live)
#8I don't like take homes, because you are able to cheat... (ask a friend, hire a fiverr, etc.) - so a brilliant solution is not very helpful to find out, if you really met an expert.
I prefer a 2 - 6 hour training day (depending on whats possible) for the best applicant, where he / she meets the team, sees how they are working and most of the people I met, were interested, because they benefit from this, too.
When you MUST provide a take home one, you could take a look at adventofcode problems as inspiration. They are fun to do, not too complicated and do not depend on a specific technology.
Re: Ask HN: How to design a good technical interview test (take home or live)
#9Good advice here: https://jacobian.org/series/work-sample-tests/ (also take a look at the series on interview questions and what makes them more or less predictive of performance)
Re: Ask HN: How to design a good technical interview test (take home or live)
#10The principles below are specifically for take-home tests, though some will still apply to technical interviews conducted live:
1. Design tests to be open-ended and demonstrate how a candidate thinks, rather than ones that drive toward a binary pass/fail outcome. Asking why a candidate worked on the pieces they did and where they would focus if given more time gets you high signal without requiring as much time from the candidate.
2. Provide clarity on the grading rubric. When candidates understand how they'll be evaluated, they'll spend less time on work that doesn't result in higher signal to the company. For example, having a candidate write some tests or talk through how they would approach testing is great! Expecting full test coverage isn't as useful.
3. Scoping is the toughest part about test design. Try to choose topics that closely mirror the day-to-day work in the role itself but that can be boiled down to a 1-2 hour task. You want to make sure that candidates who perform well are more likely to succeed if hired, but you also want to keep it short. From interviewing 50 hiring managers who designed the hiring processes at Basecamp/Medium/Mailchimp/etc, they revealed that the candidates who were ultimately hired consistently spent 2x the recommended time, which means that designing a 3-hour take-home is asking top candidates to commit 6 hours. It's a tough ask in the current hiring climate.
Here are 3 examples for inspiration:
- Full-stack role for Flow Club (YC S21) that runs virtual WeWork sessions: https://app.mightyacorn.io/flow-club/founding-front-end-engi...
- Front-end role for Playhouse (YC S21), building TikTok for real estate: https://app.mightyacorn.io/demo/playhouse-hiring-challenge
- Back-end role for zero5, a parking automation startup: https://app.mightyacorn.io/zero5/challenge-fullstack-softwar...
Happy to share more details if you have specific questions!
Source: my team spent the past 6 months designing personalized take-home tests for 20 VC-backed startups hiring founding engineers. We evaluated ~500 candidate submissions.