Live data from Hacker News

AI killed the tech interview. Now what?

kanenarraway.com

581–590 of 651 posts

Re: AI killed the tech interview. Now what?

#581
post #466
post #461

Earlier quoted context omitted.

Employers trying to hire for fit and culture has resulted in the most inhumane and counterproductive processes I've witnessed. If that's what you really want to do, let the person work for at least a month in the team.

Provisional hires (which often exist in theory but they're pretty much a formality in general) don't work for the most part. Lot of overhead for the company. And in many cases you're asking for the candidate to quit a job and possibly relocate on the possibility they'll get a new position assuming that they click in a short time interval.

Who said anything about relocation? That has to be a tiny percentage of hires.

Re: AI killed the tech interview. Now what?

#582
post #580

Earlier quoted context omitted.

I wish I could remember who, but there is a company out there who conducts tech interviews to create a pool of candidates for their customers. Pretty sure there was a post here about it, but it's lost in my ocean of unread favorites.

I personally think outsourcing hiring is the worst idea ever, and completely lose confidence in companies that do.

They’re outsourcing technical interviews, not the same thing as hiring.

(Update: technically I would guess they're only outsourcing the first few rounds. Presumably their customers will conduct further interviews.)

Re: AI killed the tech interview. Now what?

#583
post #331

Earlier quoted context omitted.

I think the argument is that for a good chunk of business work, you don't need to use the "interesting"/"deep" concepts. Sure, you'll need time to adapt to the idioms of the language you're using, but following examples you can be just as productive as others in a relatively short time.

But companies don't pay high salaries for the 80% mundane and easy tasks you do day to day. They pay for the 20% that is challenging and especially for that 1% of problems that occur only once every few months or years. If that 80% was 100% of the job then the company could pay 1/2 to 1/3rd the amount by outsourcing it.

I disagree, companies pay based on the problems you can solve to make them money or help achieve organizational goals. One of those ways can be coding, but there are many others.

Re: AI killed the tech interview. Now what?

#584
We did an experiment at interviewing.io a few months ago where we asked interviewees to try to cheat with AI, unbeknownst to their interviewers.

In parallel, we asked interviewers to use one of 3 question types: verbatim LeetCode questions, slightly modified LeetCode questions, and completely custom questions.

The full writeup is here: https://interviewing.io/blog/how-hard-is-it-to-cheat-with-ch...

TL;DR:

- Interviewers couldn't tell when candidates were cheating at all

- Both verbatim and slightly modified LeetCode questions were really easy to game with AI

- Custom questions were not gamable, on the other hand[1]

So, at least for now, my advice is that companies put more effort into coming up with questions that are unique to them. It's better for candidates because they get better signal about the work, it reduces the value asymmetry (companies have to put effort into their process instead of just grabbing questions from LeetCode etc), and it's better for employers (higher signal from the interview).

[1] This may change with the advent of better models

Re: AI killed the tech interview. Now what?

#585

Earlier quoted context omitted.

Your citation addresses it. Less than 1% of employment lawsuits are about selection criteria and employers win over 90% of them. They suggest GMA tests are _more_ defensible than other approaches. The most interesting thing in that paper is that years of experience performed so poorly. It’s in the lowest cohort. Worse than “interests” or more general “biographical data”.

There is still the reputational risk of using selection methods with widely known disparate outcomes. Other methods also have disparate outcomes, but most of the criticism is directed at IQ tests. I've heard "IQ tests are culturally biased" but never "work sample tests are culturally biased", and I'll guess that's the experience of most hiring managers too.

Somehow these reputational risks accrue to the median hiring company, but not to the global brands like PepsiCo and Proctor & Gamble that do GMA testing already. I maintain: this is a message board trope.

Re: AI killed the tech interview. Now what?

#586
It's a tricky subject, because what if people who use AI are just better together? And what if in a year from now, AI by itself is better? What's the point of hiring anyone? Perhaps this is the issue behind the problems being described, which might be mere symptoms. There are tons of very smart teams working on software that will basically replace the people you're hiring.

Re: AI killed the tech interview. Now what?

#587
post #584

We did an experiment at interviewing.io a few months ago where we asked interviewees to try to cheat with AI, unbeknownst to their interviewers. In parallel, we asked interviewers to use one of 3 question types: verbatim LeetCode questions, slightly modified LeetCode questions, and completely custom questions. The full writeup is here: https://interviewing.io/blog/how-hard-is-it-to-cheat-with-ch... TL;DR: - Interview…

A couple years ago, we were using a take home coding assignment as a hiring signal. It was a small API based off something I'd built for an internal tool. It was self-contained and relatively easy to explain. The .md file was about two pages.

I recently fed it into ChatGPT and asked it to do the assignment. It did it perfectly -- I read the code in detail and couldn't find any issues.

So custom questions are off the table now, too. We'll be using a code review instead for the next round.

Re: AI killed the tech interview. Now what?

#588
post #477

Earlier quoted context omitted.

> Company A wants to hire an engineer, an AI could solve all their tech interview questions, so why not hire that AI instead? Interview coding questions aren't like the day-to-day job, because of the nature of an interview. In an hour-long interview, I have to be able to state the problem in a way the candidate can understand, within 10 minutes or so. We don't have time for a lecture on the intricacies of voucher cal…

> In an hour-long interview, I have to be able to state the problem in a way the candidate can understand, within 10 minutes or so. We don't have time for a lecture on the intricacies of voucher calculation and global sales tax law. Proving if they are technically capable of a job seems rather silly. Look at their resume, look at their online works, ask them questions about it. Use probing questions to understand the…

Have you ever hired?

No, it's not enough. There are people out there who can talk great talk, and have great resume, but cannot do their actual job for some reason. Maybe they cannot read the code, maybe they cannot write the code, maybe they can write the code but not in the manner that keeps the rest of codebase working... I've had people like that on my team, it was miserable for all of us.

It is essential to see candidate actually write and debug code. It would be even better if we could see how the candidate deals with existing huge codebase, but sadly this kind of thing can't be easily done in a quick interview, and good candidates don't want trial periods.

Re: AI killed the tech interview. Now what?

#589

Earlier quoted context omitted.

IMO interviewing is the biggest bottleneck and if interviewing was decoupled from hiring then it wouldn't be a problem. But this requires a Guild-like organization to manage interviewing/vetting and for companies to use said Guild for hiring. The companies could then do a single team culture meeting (if they wanted) before hiring.

I wish I could remember who, but there is a company out there who conducts tech interviews to create a pool of candidates for their customers. Pretty sure there was a post here about it, but it's lost in my ocean of unread favorites.

Triplebyte is the company.

You still interview with the end companies, but technical interviews aren’t given.

Re: AI killed the tech interview. Now what?

#590

Earlier quoted context omitted.

a large company doing this (no documentation of a skills gap) who gets subpoenaed would lose 10/10 times

I'm curious, what are the recent cases that were brought up where the company lost?

Meta had one around 2020 with EEOC. The Harvard Case I mentioned above (supreme court) , Activision, Dell, a few others on the top of my mind.
Post reply on HN