@pabloadolescent : It sounds like you may have had a negative experience, and I'm certainly sorry to hear that.
@BuckRogers : We certainly respect your right to not apply if you do not like the process. That is why we are very up front about it on our application page and publicly post all the challenges in GitHub. Folks are welcome to review them prior to applying to decide with full knowledge of what may be asked. There shouldn't be any surprises. The challenges have remained largely the same for years and are in no way a method to get free labor.
We ask for the code challenges because we believe that the quality of your code is more important than the quality of your resume. The challenges are graded "blind" with each assignment given a random identifier. Each submission for each challenge is generally independently reviewed by 3 of our engineers. We feel that if we've asked you to spend time on something, we should also be willing to commit a significant chunk of our company's time into reviewing it.
The code challenges are the first step in the process, before any formal interviews, for two main reasons:
1) We want to give our engineers (not the recruiters or managers) the primary say on who their peers will be. This is ensure that our engineering culture stays strong.
2) We want to mitigate the impacts of as many unconscious biases that folks may have around any aspect of a candidate by having the "blind," skills-based review come as early as possible.
Additionally, this sort of code challenge is a far better proxy for the kind of remote work that we do than more "traditional" methods like white boarding or brain teasers.
No process is perfect. Ours certainly results in great engineers passing us by and has its fair share of false negatives. Any process will unavoidably result in the same.
We do work to ensure that our process is not an undue burden on any applicant.
- We designed the challenges so that most applicants can complete each in about two hours. If our feedback indicates that's not the case, we do make tweaks to them to adjust.
- We encourage applicants to pick only the 1 or 2 challenges that best represent their skills for the initial submission. This should minimize the "up front" costs of starting an application. The majority of applicants will end up completing 3 total, often over the course of a few weeks to amortize the time commitment.
- We try to keep the total time commitment to be, on average, roughly 8 hours. In addition to the average of three challenges, we normally have 3-4 half hour interviews (either on the phone or video chat). In this way, our process is roughly equivalent to a one-day "in person" process, but with the ability to flexibly schedule things to what works best for the candidate (and avoids additional overhead for time to travel to/from a location).
Our current process is the result of much experience and reflection, but I'll spare everyone the dissertation on it all. Hopefully the above provides some useful context for those considering applying.