Rather than showing the top ten for statistical significance, wouldn't it make more sense to use PCA on the ratings and show each component's top 10 instead?
We analyzed 100K technical interviews to see where the best performers work
91–100 of 234 posts
Re: We analyzed 100K technical interviews to see where the best performers work
#92Does this happen in any other field? If I was a doctor and wanted to work at some other company, would I need to study the MCAT every year in order to pass a screening interview based on one possible question? What's the equivalent in other fields? Closest I can think of is an acting audition but even then they give you the script beforehand. I'm beginning to think that the industry somehow settled on this approach not so much as a skills verification process but, by making the process so onerous on the candidate, talent retention is a lot easier.
Re: We analyzed 100K technical interviews to see where the best performers work
#93I'm glad to see this getting roasted in the comments, as it's a really good example of how companies put out self-serving pseudo-statistical nonsense in an effort to promote themselves. There's no effort to quantify what "technical" or "communication" skills are - these are left to the interpretation of a the interviewer. It makes no effort to show where these engineers are going, what the interview process they're c…
Author here. Yes, the skills are left to the interpretation of the interviewer, but most of our interviewers are senior engineers at FAANG. We've done quite a bit of work internally to make sure your interviewers are well calibrated, and we have a living calibration score for each one (calibration is based on how the interviewees they interview end up doing in real interviews). The interviews in question are a mix of…
> Also, if I ever use "rockstar" or "ninja" in my posts, I hope someone finds me on the street and punches me in the face. I'd deserve it.
Are you sure you'll never find yourself blogging about bands' flamboyant frontmen (or frontwomen), or covert agents in feudal Japan?
Re: We analyzed 100K technical interviews to see where the best performers work
#94I'm glad to see this getting roasted in the comments, as it's a really good example of how companies put out self-serving pseudo-statistical nonsense in an effort to promote themselves. There's no effort to quantify what "technical" or "communication" skills are - these are left to the interpretation of a the interviewer. It makes no effort to show where these engineers are going, what the interview process they're c…
Author here. Yes, the skills are left to the interpretation of the interviewer, but most of our interviewers are senior engineers at FAANG. We've done quite a bit of work internally to make sure your interviewers are well calibrated, and we have a living calibration score for each one (calibration is based on how the interviewees they interview end up doing in real interviews). The interviews in question are a mix of…
Is there any statistical reason I should assume an interviewer at FAANG has some kind of special insight into what good technical and communication skills are? Were any attempts made to adjust for demographic biases? Is it possible, for instance, that Dropbox engineers are disproportionately taller white dudes with nice hair? I know that sounds a bit unserious, but all of those factors would increase the likelihood of a higher score.
I appreciate that you never used "rockstar" or "ninja". That dig was a bit unfair of me.
Re: We analyzed 100K technical interviews to see where the best performers work
#95> There’s also the issue of selection bias. Maybe we’re just getting people who really feel like they need practice and aren’t an indicative slice of engineers at that company. Or that your interview preparation platform prepares candidates better for Dropbox's interview process than it does for Microsoft's. Or that the people who were confident in their interview skills for Facebook decided not to use your platform.…
Author here. The data is mostly drawn from how people who work at these companies do in mock interviews rather than how our users do in real interviews with these companies.
You should probably change the title of the blog as most gainfully employed people interpret 'best performers' as people that are very good at performing their job and/or trained for a specific Circus Act.
Something like "We analyzed 100K technical interviews to see which companies employ the people that we feel best performed in our mock interviews" would be more authentic
Re: We analyzed 100K technical interviews to see where the best performers work
#96I'm glad to see this getting roasted in the comments, as it's a really good example of how companies put out self-serving pseudo-statistical nonsense in an effort to promote themselves. There's no effort to quantify what "technical" or "communication" skills are - these are left to the interpretation of a the interviewer. It makes no effort to show where these engineers are going, what the interview process they're c…
Author here. Yes, the skills are left to the interpretation of the interviewer, but most of our interviewers are senior engineers at FAANG. We've done quite a bit of work internally to make sure your interviewers are well calibrated, and we have a living calibration score for each one (calibration is based on how the interviewees they interview end up doing in real interviews). The interviews in question are a mix of…
I suppose Douglas Hofstadter is en route to punch you right now
Re: We analyzed 100K technical interviews to see where the best performers work
#97All this post says is that among the candidates who scored highest on this model, a higher proportion worked at these companies. But there's no inverse analysis: of people who worked at these companies, how predictive was that overall of a higher score on this particular assessment? 'Our five highest scores ever were all people who wanted to leave FooCo' tells you little about the overall quality of FooCo employees.…
No you're misinterpreting it. Go back and reread the graphs.
Ah, you're right. The graphs don't actually show the data they describe their methodology as collecting. That fills me with confidence.
Re: We analyzed 100K technical interviews to see where the best performers work
#98The fallacy to these findings and conclusions is that you're measuring processes of your own brand and flavor; the biases are so deep you likely don't recognize them. 'technical', 'problem solving', 'communication' these are all of your own construction, setting, presentation and scoring. Look at the failings of the IQ test for example. I'm curious if people just really need to learn the talk and the walk of a silico…
> The fallacy to these findings and conclusions is that you're measuring processes of your own brand and flavor
> Look at the failings of the IQ test for example.
You are mis-interpreting the point of all of this. This website does practice interviews. It doesn't measure how good of an engineer someone is, or how smart they are, and they don't claim to do so.
Instead, it measures how good they are at tech interviews. And for that purpose, the study works well enough.
Re: We analyzed 100K technical interviews to see where the best performers work
#99I'm glad to see this getting roasted in the comments, as it's a really good example of how companies put out self-serving pseudo-statistical nonsense in an effort to promote themselves. There's no effort to quantify what "technical" or "communication" skills are - these are left to the interpretation of a the interviewer. It makes no effort to show where these engineers are going, what the interview process they're c…
"There's no effort to quantify what "technical" or "communication" skills are - these are left to the interpretation of a the interviewer." And yet, these scores are measuring something and averaged across tens of thousands of technical interviews, you have enough statistical power to average out the particularities of each interviewer. I'm sorry you find the results repugnant, but the results are what they are. And…
It's not the results I find repugnant, it's the assumption that the results have any real world validity. The _something_ they're measuring is as likely to be demographic biases as it is technical or communication skills.
Re: We analyzed 100K technical interviews to see where the best performers work
#100Kudos to interviewing.io to share this analysis. I agree with the many issues in methodology and analysis that others have raised here, and I agree there's a risk that a face-value reading of the blog post is highly misleading. But this is true for all data, and poo-pooing the analysis without crediting the sharing just leads to less sharing. To be clear, I'm supportive of the criticism, but let's also give credit where it's due.
Technical interview performance is a high stakes field for which almost all data is cloistered in individual hiring companies or secretive gatekeepers. In my mind, all efforts, even imperfect ones, to share data is a great step here. We should encourage them to continue to share, including pursuing options to anonymize the raw data for open analysis. The field deserves more transparency and open discussion to push us all to be better.