Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We analyzed 100K technical interviews to see where the best performers work

blog.interviewing.io

Post reply on HN
Print view View original

111–120 of 234 posts

Re: We analyzed 100K technical interviews to see where the best performers work

#111

Post by CobrastanJorji »

leeny wrote: post #88
kieselguhr_kid wrote: post #44

I'm glad to see this getting roasted in the comments, as it's a really good example of how companies put out self-serving pseudo-statistical nonsense in an effort to promote themselves. There's no effort to quantify what "technical" or "communication" skills are - these are left to the interpretation of a the interviewer. It makes no effort to show where these engineers are going, what the interview process they're c…

Author here. Yes, the skills are left to the interpretation of the interviewer, but most of our interviewers are senior engineers at FAANG. We've done quite a bit of work internally to make sure your interviewers are well calibrated, and we have a living calibration score for each one (calibration is based on how the interviewees they interview end up doing in real interviews). The interviews in question are a mix of…

FAANG interviewer here. I've conducted many hundreds of interviews for multiple FAANG companies. The totality of my training in how to interview is about 4 hours (when I combine the training of each company). I have zero confidence in the usefulness of calibrating my answers and expect that neither I nor anyone else who does this would reliably score the same person with the same score most of the time, outside of a small percentage of outlier candidates.

Re: We analyzed 100K technical interviews to see where the best performers work

#113

Post by gcheong »

WYepQ4dNnG wrote: post #55
bckr wrote: post #20

Earlier quoted context omitted.

I think interview practice with real interviewers and expert feedback is an excellent investment.

My strategy is simple: I do interview practice by simply interviewing with companies that I am not really interested in joining. That allows me to practice not only the technical interview but also all other steps involved. Last time I interviewed, I did 7 of these interviews, targeting: startups, pre-ipo, mid public companies, 2 of the FAANG I would never work for ... I did that while prepping for interviewing at th…

I think the value is mainly in the feedback you get which can help if you have a specific problem you might not be aware of. The problem with just going out on interviews is that you never get any real feedback so you are left to your own analysis as to why a certain interview didn't result in getting to the next step or an offer which, in my experience, is kind of like trying to find a needle in a haystack in a dark barn without a flashlight.

Re: We analyzed 100K technical interviews to see where the best performers work

#114

Post by psyc »

kieselguhr_kid wrote: post #44

I'm glad to see this getting roasted in the comments, as it's a really good example of how companies put out self-serving pseudo-statistical nonsense in an effort to promote themselves. There's no effort to quantify what "technical" or "communication" skills are - these are left to the interpretation of a the interviewer. It makes no effort to show where these engineers are going, what the interview process they're c…

While I agree that this particular exercise is riddled with problems, I simply cannot image Hacker News rolling over and accepting evidence-based answers to questions of this nature, regardless of where the data came from or what the methodology was.

Re: We analyzed 100K technical interviews to see where the best performers work

#115

Post by visarga »

gcheong wrote: post #92

Does this happen in any other field? If I was a doctor and wanted to work at some other company, would I need to study the MCAT every year in order to pass a screening interview based on one possible question? What's the equivalent in other fields? Closest I can think of is an acting audition but even then they give you the script beforehand. I'm beginning to think that the industry somehow settled on this approach n…

> I'm beginning to think that the industry somehow settled on this approach not so much as a skills verification process but, by making the process so onerous on the candidate, talent retention is a lot easier.

There are plenty of applicants who just can't do the job, and the cost of restarting the search all over again is high. It's important for a team to vet their new hires.

Re: We analyzed 100K technical interviews to see where the best performers work

#116

Post by lamontcg »

Going to be real interesting to see how I do at technical interviews whenever I decide to jump back into the job market.

At the job I'm leaving tomorrow, I just did a 2 hour long video session / training where I started by teaching how to read call graphs, and that led to over a hour of me trying to sort out a horrible performance issue in real time.

The problem, solution and the iterated debugging to deal with all the edge conditions that the extensive unit tests called out (and I wrote all the unit tests that blew up, so I get to take credit for all that -- although I also wrote the bug I fixed) should show that I'm very high functioning engineer. And I had identified the problem previously at a higher level and had a fix that papered over the problem, but during the video I correctly figured out that the real source of the problem was deeper in the code, and had existed before the change which surfaced the problem, and managed to do a data-driven analysis to track down the perf bug and go from 15% of CPU time in one subsystem to 1% of CPU time in the same subsystem for a 15x speedup on my problem (and probably closer to a 90x speedup for the customers who were reporting it--including a large customer everyone is familiar with here due to headlines they're involved in).

Meanwhile I forgot that it was obj.send(:method, *args) in ruby and tried to obj.call(:method, *args) and had to look that up because my brain was derping a bit on that, and the night before I forgot it was JSON.generate in ruby and not encode/decode and just in general my brain is a mash of too many different programming language syntaxes. At one point I caught myself trying to use `%` for comments because I had been doing Matlab writing an Iterated Chebychev Picard Method IVP ODE solver the prior weekend. If I can't work with the command line or an IDE and with google I'm just going to be a mess of trivial mistakes due to crossed wires.

I've also never reversed a linked list in my life and the correct answer to that question is probably to never use a linked list due to TLB cache thrashing at the very least.

Re: We analyzed 100K technical interviews to see where the best performers work

#118

Post by kjeetgill »

rootusrootus wrote: post #61
kieselguhr_kid wrote: post #44

I'm glad to see this getting roasted in the comments, as it's a really good example of how companies put out self-serving pseudo-statistical nonsense in an effort to promote themselves. There's no effort to quantify what "technical" or "communication" skills are - these are left to the interpretation of a the interviewer. It makes no effort to show where these engineers are going, what the interview process they're c…

> It perpetuates the myth that there's something really special about Silicon Valley engineers Does this myth still have much traction? If anything, my general regard for engineers in the bay area has steadily declined in the last few years. There are so many really worthless folks who have only figured out how look like they have a clue, but go any deeper and they flail. I know I'm painting with a broad brush, and t…

Hey, it's not our fault! Where else are posers gonna flock to to pose? I swear, even with all the remote work, more than 95%* of the people that worked here 5 years ago still do. We live here.

* made up statistic.

Re: We analyzed 100K technical interviews to see where the best performers work

#119

Post by dasil003 »

stale2002 wrote: post #101
kieselguhr_kid wrote: post #99

Earlier quoted context omitted.

It's not the results I find repugnant, it's the assumption that the results have any real world validity. The _something_ they're measuring is as likely to be demographic biases as it is technical or communication skills.

> it's the assumption that the results have any real world validity Of course it has real world validity. It has real world validity regarding how well people are likely to do in tech interviews. You can feel free to say that the way the tech industry does interviews is bad, or biased, or has all sorts of problems. But having vague measurements, such as "technical" or "communication" skills sounds very accurate to ho…

The moral outrage is because of Goodhart's Law.

Over the last 22 years I've interviewed hundreds, hired and directly managed probably close to 100 individuals in widely varying environments, ranging from a 25k employee state university, to startups of various shapes and sizes, to the hottest SV IPO of 2020.

The reason for interview practices to be the way they are is to raise the floor for FAANG, who have high internal complexity and high salaries, leading to a very fat hiring funnel with a high risk of turning into dead weight in "rest and vest" mode once they get inside.

However humans are smart, and interviewers are lazy, so inevitably the people who optimize for this process start beating out more able engineers who don't have the time or inclination to jump through these hoops. In my experience, the proportion of really good software engineers is roughly equivalent across all companies with baseline competent technical leadership. FAANG does have a lot of the outliers on the high end, but they also have a lot of folks who can't tie their own shoes without the support of world-class tooling, infra support, and technical design guidance that those companies surround them with.

Re: We analyzed 100K technical interviews to see where the best performers work

#120

Post by kieselguhr_kid »

leeny wrote: post #110
kieselguhr_kid wrote: post #94

Earlier quoted context omitted.

Is there any statistical reason I should assume an interviewer at FAANG has some kind of special insight into what good technical and communication skills are? Were any attempts made to adjust for demographic biases? Is it possible, for instance, that Dropbox engineers are disproportionately taller white dudes with nice hair? I know that sounds a bit unserious, but all of those factors would increase the likelihood o…

no, we didn't adjust for demographic biases because, look, we're an anonymous platform. we periodically survey our users to get demographic data, but it's not something we ask in the app because we've never been able to resolve "tell me your race & gender" with "hey we're anonymous, and you'll be judged on your performance and nothing else". last thing i want is to perpetuate stereotype threat inadvertently. it's pos…

The point about stereotype threat is valid, although it also means that we're unable to get any insight into a major axis of interview performance.

I think what we have here is an attempt to imply, consciously or not, a causal link between interview success and previous/current employment. But without drilling into the other factors underlying their success, we get a lot of noise and not enough signal. Couple that with the continued mythologizing of FAANG greatness, and you get an article that perpetuates two of the more toxic notions in tech: FAANG is the top of a pyramid and talent is concentrated in a handful of companies. Neither are true, and neither are probably your intent, but that's how this reads.

Post reply on HN