201–210 of 282 posts
Earlier quoted context omitted.
I'd say it's the exact opposite. There are hordes of unqualified people applying to every software dev role imaginable regardless of what you put in the job description or requirements. The tests are there because people are good at lying but bad at faking skills.
Seriously, if you've never hired before, you have no idea how bad this can get. Here's [1] our practice coding problem. It's quite similar to the one we use on our interview, and not too far from the one Triplebyte used in the past (ours is tuned to be slightly harder at the beginning and slightly easier at the end). The vast majority of candidates, even with some reasonable pre-filtering, do not get past the first s…
Would be interesting to hear Rachel's take on all the contract recruiting (?) firms that claim to do the same thing, test candidates and then provide them to employers- just on remote contracts for temp work. The OG in this space is Toptal, but there seems to be a lot of VC money in a bunch of upstarts in this space. They all say 'we screen our developers through an extremely rigorous yada yada process'. They tend to…
-----
(From https://www.toptal.com/top-3-percent)
> We also test each applicant's technical knowledge and problem-solving ability through various assessments.
"Various assessments" is about as maximally vague as you can be. Imagine if I asked you how you'd build a back end and you went "oh, you know, various technologies, um, APIs, probably some functions" - that's the level we're operating on here.
Here's how you do it not vaguely: Otherbranch's assessment is, currently, a 90 minute call with an engineer. It's synchronous over video and screen-share and split into 3 subsections: coding, knowledge, and system design. Coding is a small app in the console, you can find an example problem at https://www.otherbranch.com/practice-coding-problem . Knowledge is a mix of CS/algos, full-stack web development, and "deep tech" (systems-level/security/internals). System design is really standard "build this basic business system and scale it up". If you want more depth on our grading, I can show you that, although I don't share our actual questions verbatim. I did the structure and I've led large-scale assessments before, and our interviewers have done thousands of hours of interviews and several are ex-FAANG.
Now, granted, we're in one domain and they're in many, so specifics may be hard to produce here. (And they may even be bad sales practice, so there might be non-malicious reasons for not including them.) But it still catches my eye.
> we typically only advance candidates with exceptional results in this phase.
26.4% of candidates pass basic English screening, according to the numbers next to the previous claim. Of those, they claim, 7.4% pass this step. Note that no screenings for actual skills have occurred prior to this step. If we assume that domain skill and English screening are weakly correlated (this is probably a bad assumption, literally any two skills anywhere have positive correlations), they're passing roughly 25% of applicants by skill. Let's generously say 15-20%. And the 80-85th percentile of applicant is horrible. So I just don't buy "exceptional results" here.
> Each candidate is interviewed by Toptal screeners who are experts in their functional domain.
Again, vague. Interviewed for how long? With what criteria? How are you determining that they are "experts in their functional domain"?
> Each candidate is assigned a test project to evaluate whether they can "walk the walk." Test projects take 1-3 weeks are comprehensive and provide real-world scenarios for candidates to demonstrate their competence, thoroughness, professionalism, and integrity.
This part looks better! Until you look at the numbers next to it, which indicate a 90+% pass rate for this phase (and still a roughly 10% pass rate from "people who cleared basic English language skills".
It's easy to say you're rigorously assessing people. But when they're so evasive about the details, I get suspicious.
-----
I'm not saying Toptal isn't better than nothing. I'm not an expert on international hiring, and they seem to be doing pretty well as a business. And this vagueness may very well be a feature, not a bug - my verbosity gets me into trouble sometimes when I write copy. In fact, this very post demonstrates a good reason NOT to be specific, because armchair experts will pick your exact numbers apart without the appropriate context. (I justify myself by saying that this is, in fact, my area of expertise.) But if you want my snap take, there it is.
Toptal people reading this thread, feel free to drop in with details if you got em!
Earlier quoted context omitted.
Seriously, if you've never hired before, you have no idea how bad this can get. Here's [1] our practice coding problem. It's quite similar to the one we use on our interview, and not too far from the one Triplebyte used in the past (ours is tuned to be slightly harder at the beginning and slightly easier at the end). The vast majority of candidates, even with some reasonable pre-filtering, do not get past the first s…
that's actually pretty great, but it brings up another issue I have with the state of tech interviewing - it focuses on being able to write code fast . the more senior you get, the more you tend to focus on depth, taking your time to think over the problem and write a good robust solution rather than banging out code fast, so coding up a solution in 25 minutes versus an hour is not really a good test of what the comp…
First, you massively underestimate the range of coding speed you see in an interview. The slowest programmers weren’t senior people who were out of practice. (I interviewed plenty of them). It was people who just seem bad at programming. Like, so bad it takes them 25 minutes to make a hello world program run. (In their favorite language, on their computer and with full access to the internet during the test).
A 2x programming speed difference would have rarely changed the outcome of our overall assessment.
Second, there was an aspect of triplebyte’s interviewing process that I’d love to see replicated elsewhere in the industry that resolves this. And that is, we should be assessing debugging ability. At triplebyte we gave candidates a smallish program (few hundred lines) with 4 bugs and a failing test case for each one. The candidates had half an hour to fix as many of the bugs as they could.
Watching people debug was fascinating.
One clear pattern that emerges is exactly what you are predicting. Smart kids right out of school were great at the programming section. But it was always the more senior engineers who smashed the debugging section. Junior engineers would get lost in the weeds and struggle to get very far in the time we gave them. Some of the senior people I interviewed dived straight in, and even found some bugs we didn’t even know about in our own test.
It seems to me that being able to read unfamiliar code and fix bugs in it is a hard to learn skill that matters on the ground. And frankly I suspect it’s more useful skill than a lot of leetcode problems. I’d rather hire someone who’s amazing at debugging than someone who’s amazing at data structures. Well, I suppose I want one of each on my team.
If I was ever making a programming test, this is something I’d include for sure.
I always find it strange that people who talk about tech interviewing inexplicably overlook what seems to me to be a core defining characteristic: they are highly traumatic. You take some poor bastard and have him struggle at coding puzzles in front of someone he very much wants to impress and then watch as he fails miserably. They are left feeling like they are biologically inadequate to their job. It's a direct ass…
Like, I'm enough of a leftie to agree with the idea that one's ability to contribute in the workplace shouldn't determine your ability to live a decent life. But that doesn't mean companies should hire someone who can't do the job, it means being unemployed shouldn't be a virtual death sentence, which isn't fundamentally a problem of hiring processes.
Earlier quoted context omitted.
That made me actually laugh out loud. Maybe we should try for some Last Starfighter growth hacks.
Just call the upper levels the Star League. ("You have been recruited by the Star League...")
Now I feel like I should name all my strategic initiatives like I'm a Dragonball Z character. Maybe I should stop posting in this thread and go...lie down or something. That feeling can't possibly be good.
Earlier quoted context omitted.
You consider that a "data structure question"? Honest question - I would (do) characterize it as a sort of "fizzbuzz+" that is deliberately NOT data-structure-y, and I'm surprised by this response. Can you give an example of short coding tasks you would consider not data-structure-y? (For the record, we do ask about DBs and system design in other sections of the interview. The coding is one portion of three for the i…
Minesweeper is all about storing data about the board and displaying the numbers is all about running through data. Seems very data structurey to me but maybe that's non heavy dev side coming through. Maybe I'm just thinking too much problem, overloaded my small attention brain and misread it. I wouldn't call Fizzbuzz data structure-y however. I'm also mostly outsider looking in. Never worked at FAANG and working wit…
A “data structure problem” would involve more exotic data structures, usually of the kind the candidate has to implement themselves. For example, b-trees, heaps, skip lists, and so on.
The reason a lot of people don’t like custom data structure questions is that they come up rarely in most people’s jobs. Lists, structs and enums on the other hand are used everywhere. Your programming job will almost certainly require you to understand them.
Earlier quoted context omitted.
You consider that a "data structure question"? Honest question - I would (do) characterize it as a sort of "fizzbuzz+" that is deliberately NOT data-structure-y, and I'm surprised by this response. Can you give an example of short coding tasks you would consider not data-structure-y? (For the record, we do ask about DBs and system design in other sections of the interview. The coding is one portion of three for the i…
the problem is embedding game logic (game rules) into the matrix, which is a data structure, hence it is a data structure question
Earlier quoted context omitted.
this is some straight up Ender's Game shit
That made me actually laugh out loud. Maybe we should try for some Last Starfighter growth hacks.
For anyone hunting for more examples, I'd suggest this [Caution: Time-sucking wiki] TvTropes listing [0].
[0] https://tvtropes.org/pmwiki/pmwiki.php/Main/AndYouThoughtItW...
I always find it strange that people who talk about tech interviewing inexplicably overlook what seems to me to be a core defining characteristic: they are highly traumatic. You take some poor bastard and have him struggle at coding puzzles in front of someone he very much wants to impress and then watch as he fails miserably. They are left feeling like they are biologically inadequate to their job. It's a direct ass…
Stress I'll grant you. But what's your alternative to selectiveness? Is your expectation that companies, who are spending a substantial amount of money to hire someone, should not try to hire the best person they can get for their buck? Like, I'm enough of a leftie to agree with the idea that one's ability to contribute in the workplace shouldn't determine your ability to live a decent life. But that doesn't mean com…
I just think the fundamental problem here is not procedural as the post seems to suggest - but rather social-psychological. Making the experience less painful to the losers is the key problem to solve.
That would fix the candidate pipeline problem because people would be less terrified of failure.
I don't know how to solve it.
To quote Leonard Cohen:
It's coming from the sorrow in the street
The holy places where the races meet
From the homicidal bitchin'
That goes down in every kitchen
To determine who will serve and who will eat