Earlier quoted context omitted.
OP here. Agree with this sentiment. Even I would not use cull.io to have candidates build a complex web app that would take days or even hours to build. I would use it as a tool to "Filter" candidates, not "Select" them. (It's amazing how the simplest of a coding challenge would filter all non-coding coders) thanks for the review.
I've never been on the other(interviewer) side of the hiring process and I was on another side only once during the last decade, so I might not have enough insight, but I really don't understand non-coding coders filtering per se, be it your method or FizzBuzz... When someone says about their years of experience in coding in their CV, is it a common case that they're lying?
I, somewhat cynically, divide job seekers into four categories:
1) the qualified (I know what I can do, and I can do what you need)
2) the unqualified (I know what I can do, but I can't do what you need)
3) the deluded (I honestly think I'm qualified, but I'm not)
4) the liars (I'm trying to get the job by any mean necessary).
I only want (1).
(2) are easy to deal with - you just have to know what you need.
(3) are common and mildly annoying - especially if you're a nice person and feel the need to explain where they are deluded :-)
(4) are not common - but are exceptionally annoying and often hard to sort out
Writing a really good job spec / advert will get rid of most of (2) for you, and help remove all but the most deluded from (3).
Filtering resultant CVs will usually get rid of the rest of (2), and a bit more of (3).
A quick chat on the phone will get rid of any remaining (2) and a big chunk of (3).
Simple tests similar to FizzBuzz will cull a surprising amount of (3) and (4).
Interviewing - especially good accomplishment based questions will tend to filter out all of (3) and a big chunk of (4).
Which, unfortunately, occasionally leaves some of (4) who seem to sometimes put more effort into deceiving me than it would take to actually learn the damn skills in the first place. Which is why I really like having some kind of solid work "with the team" session for at least a couple of hours. Pay 'em for it too if at all possible.