111–120 of 190 posts
Studies [1] show that a work sample test is the best predictor of candidate performance on the job, which is why many software engineering teams use take-home tests as one step in their hiring process. But designing an effective test is difficult and time-consuming. For example, candidates are reluctant to complete tests that are too long or not engaging enough. But make them too short and teams won’t get the signal…
Earlier quoted context omitted.
You awarded 5 of them, or 3 of them? If you're only paying the people you hire, that's not paying people for their time. I mean sorry, but if you did a good job, we're going to make you an offer such that you won't need to quibble about $500. The entire difficult part is what you're asking from people you don't hire!
Received 5 PRs, awarded the top 3. You can actually see them https://app.algora.io/algora/challenge/pulls/5 We clarified it in our bounty issue that only the best-winning submissions would get the bounty. Other folks have chosen to award all PR submissions regardless of hiring outcome. The format is up to the company hiring. You're making a great point! It's exactly about what we ask from people who don't get hired.…
As an interviewer, I believe take home tasks are unfair and also poor return on investment. Take home tasks incentivise candidates to burn as much time as possible on the task. Consequently, they are a test of who is willing to put a week of work even when the instructions say it should not take more than 3 hours to complete. I prefer pair programming with the candidate on the premise that if I ask you to spend 2 hou…
Real world challenges needs time to think, often time to refactor after initial working prototype. A take home allows that.
Also the candidate is already relaxed in the pair programming session because it's a familiar codebase. You'll get signal if they didn't write it or wrote something they didn't quite understand.
It's painful to imagine how many teams miss out on candidates because their take-home or technical interviews are not extracting the right signals. Having access to a library like this will save so much time on getting new ideas and iterating. At the very least, I hope it saves the effort of running into useless leetcode libraries.
Or it'll just cause the leetcode bar to be moved to fit the tests with "stronger signals". Goodhart's Law will basically apply to any exam under capitalism: "When a measure becomes a target, it ceases to be a good measure." If these jobs didn't offer more compensation and prestige than other positions, matching candidate competency to job requirements would be much more achievable, but as it is now, it's an adverseal…
Whatever is closest to the real skills you need will makes it harder to improve at the measure without also improving at the real job skills. At least that would be the dream.
Earlier quoted context omitted.
How would such a cap be enforced? If it's due 120 minutes from the time it's given, it's no longer a take home interview, it's a remote interview.
It doesn't need to be proctored. It would be easy to start a remote clock and the solution needs to be in by x minutes after the start. It can even be pretty generous. The idea is presumably not to see how fast they can sprint but just put some time-boxing in place so some people aren't taking days. People can get help of course but that's going to be the case absent effectively a remote proctored assignment. And now…
I would vastly prefer an "in-person" interview in which I wasn't expected to speak to the interviewer to the in-person interviews we actually have. There is a world of difference.
As an interviewer, I believe take home tasks are unfair and also poor return on investment. Take home tasks incentivise candidates to burn as much time as possible on the task. Consequently, they are a test of who is willing to put a week of work even when the instructions say it should not take more than 3 hours to complete. I prefer pair programming with the candidate on the premise that if I ask you to spend 2 hou…
If you give every person on the street 2 hours of your time, you wouldn't be able to do your day job. I don't like take home tests either, but the point of them is removing the bad candidates from the pipeline so that people who interview them (often ICs, and these are some of the biggest expenses within the company) don't waste time on someone that is unlikely to be qualified. Personally when I'm interviewing and I…
All kidding aside, the efficient thing about resumes is that you write them once and use them everywhere. Coding samples are inefficient because you have to take 4h to write one for each company.
Edit: the other problem with takehomes is that it self-selects the people who have scarce-enough opportunities that they find it worth their time to do them.
Earlier quoted context omitted.
>I think everyone is too focused on technical prowess >and will tell you nothing about quality of code delivered. These statements seem a little contradictory or I'm misunderstanding the context.
Not being too focused on technical prowess doesn't mean being oblivious to code quality, so no I don't think they are contradictory.
If anything it at least showcases the importance of clear communication and how easily things can get lost in the details.
Studies [1] show that a work sample test is the best predictor of candidate performance on the job, which is why many software engineering teams use take-home tests as one step in their hiring process. But designing an effective test is difficult and time-consuming. For example, candidates are reluctant to complete tests that are too long or not engaging enough. But make them too short and teams won’t get the signal…
One thing I wish I could see but probably isn't compilable is how much an impact the test has on the final decision or what stage it's at. Lots of the comments are pointing out a 2-4h takehome is a poor value prop from the candidate's side compared to a 1h leetcode which transfers better across companies. (I'd agree with this.) But a 1h leetcode is usually just a first hurdle to what's often another 1-3 rounds, while…
While it's not compilable, I did reach out to a few of the 5-star test designers and asked them this question. How teams use their tests (both in weight and stage) varies a lot, but the well-designed tests usually featured as a central component of the hiring process. They were often given at the stage right before the final round of interviews, but occasionally earlier. The 5-star test designers I spoke with weighted the test heavily because it: 1. helped them reduce bias, identifying great candidates even when they didn't have the backgrounds they expected 2. gave them a foundation for the future interviews. It's less stressful on candidates when they're already familiar with the code being discussed live (because they wrote it!)
Anecdotally, on-the-job performance of the candidates they ultimately hired seemed to correlate with performance on the test. I realize this is very hard to compare against other evaluation approaches though!
Earlier quoted context omitted.
That would actually be a huge red flag to me for hiring. I don't want an architecture astronaut who is never going to ship anything. An 80% solution when I need it is infinitely more useful than a 100% solution that's months late.
And I'd rather somebody admit that they took longer, say 4 hours to get to the 80% because they were a bit rusty, their IDE on their private machine wasn't as expected (e.g. Intellij CE vs the Ultimate at work), they had to look up how to bootstrap a service from 0 because they hadn't done that in ages even at work etc. than to under deliver a 20% solution in the 2 hours asked. Limits are fine if you ask me. Give me…