I used TripleByte to recruit people for a startup without paying, by asking people to sign up for the test and screenshot the answers and collecting the scores at the end. I already had an inbound funnel, I just liked your test when I saw it the first time.
In 2015, candidates with the same schooling scored about 20 percentage points lower than 2020, the last candidates I used the test for. The number of questions doubled, and then the test got much easier, by eliminating more challenging programming questions and replacing them with questions with giveaway context. By comparison, according to my data, about N=43, until about 2018 being a senior versus a junior in college CS programs is worth about 10 percentage points on the test; going to Harvard instead of Berkeley is also worth about 10 percentage points. In 2020, the last tests had no predictive features.
I stopped using the test, because it became too easy and too noisy to be informative.
I recognize some of the coded language in the blog post. There are definitely more lucrative opportunities in recruiting for DEI. I don't know if it will last. If you're still jittery about the public-profile-by-default thing, which by the way, was totally irrelevant and overblown IMO, this may not be a pivot for you.
One thing I see in the data is that at MIT, women and men performed the same, controlling for seniority. This wasn't true at the 3 other universities that produced enough data to measure.
That said, what really is the best way to hire candidates? I'm not convinced having binders full of engineers is special, there are almost always more candidates than jobs, at least 5:1, in every non-credentialed industry vertical. Anyone who has worked at a jobs (or indeed any matching platform, like the Common App or Tinder) knows that.
Then there's this long thing about asymmetries or whatever, warble garble about missing information... It has never, not once been my experience that someone seeking a subordinate role at a typical private company with preferences like "pair programming" or whatever have ever been better than someone with no preferences at all.
Maybe it helps to engage in the vanity of whatever trendy workplace trend is hot for whatever vertical. But like, if you're being intellectually honest, if you thought pair programming was important, you'd pair program at TripleByte, but you don't, you know in your heart of hearts none of that shit matters, so why are you putting stuff like that into your search system?
Indeed and ZipRecruiter are ad arb companies. They don't care. Private universities lead, not lag, DEI at giant companies, so it's hard to see how to compete against them in that core business. It will still come down to a real defensible opinion.
Do you have more valuable inventory than ZipRecruiter for DEI candidates? Who knows. What an uninteresting question. Apple also hires people who just make shit up on their resumes, I know two - though they weren't engineers.