Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Broke Interviews

yusufaytas.com

Post reply on HN
Print view View original

51–60 of 132 posts

Re: AI Broke Interviews

#51

Post by Gigachad »

TrackerFF wrote: post #18
vasilzhigilei wrote: post #14

Interviews should be in-person.

In every practical sense, online interviews are the part of the early screening process. The sheer amount of applicants means that you need to do some filtering before inviting people to do on-site interviews.

If you make the first interview in person, most people filter themselves out because they aren’t in the country or can’t be bothered.

Do a first phone screening to agree on the details of the job and the salary, but the actual knowledge testing should be in person.

Re: AI Broke Interviews

#52

Post by nradov »

storus wrote: post #48

Universities and education overall also had their foundation detonated by AI. Some Stanford classes now do 15 minute tricky exams to reduce the chance of cheating with AI (it takes some time to type it so the point is to make the exam so short that one can't physically cheat well). I am not sure what the solution for this mess is going to be.

Several possible solutions:

1. Strict honor code that is actually enforced with zero tolerance.

2. Exams done in person with screening for electronic devices.

3. Recognize that generative AI is going to be ambient and ubiquitous, and rework course content from scratch to focus on the aspects that only humans still do well.

Re: AI Broke Interviews

#53

Post by Gigachad »

torben-friis wrote: post #7

I don’t understand how offline interviewing is needed to catch ai use, not counting take homes. Surely just asking the candidate to lean a bit back on the web interview and then having a regular talk without him reaching for the keyboard is enough? I guess they can have some in between layer hearing the conversation and posting tips but even then it would be obvious someone’s reading from a sheet.

There are interview cheating tools which listen in on the call and show a layer over the screen with answers which doesn’t show on screen shares.

So you’d only be going off how they speak which could be filtering out people who are just a bit awkward.

Re: AI Broke Interviews

#54

Post by [deleted] »

mtneglZ wrote: post #31

I still think how many golf balls fit in a 747 is a good interview question. No one needs to give me a number but someone could really wow me but outlining a real plan to estimate this, tell me how you would subcontract estimating the size of the golf ball and the plane. It's not about a right or wrong answer but explaining to me how you think. I do software and hardware interviews and always did them in person so we…

[deleted]

Re: AI Broke Interviews

#55

Post by Gigachad »

highfrequency wrote: post #28

If AI can solve all of your interview questions trivially, maybe you should figure out how to use AI to do the job itself.

The questions were just a proxy for the knowledge you needed. If you could answer the questions you must have learned enough to be able to do the work. We invented a way to answer the test questions without being able to do the work.

Re: AI Broke Interviews

#56

Post by dagmx »

I’ve mentioned it before, but it’s not just that people “cheat” during interviews with an LLM…it’s that they have atrophied a lot of their basic skills because they’ve become dependent on it.

Honestly, the only ways around it for me are

1. Have in person interviews on a whiteboard. Pseudocode is okay.

2. Find questions that trip up LLMs. I’m lucky because my specific domain is one where LLMs are really bad at because we deal with hierarchical and temporal data. They’re easy for a human but the multi dimensional complexity trips up every LLM I’ve tried.

3. Prepare edge cases that require the candidate to reconsider their initial approach. LLMs are pretty obvious when they throw out things wholesale

Re: AI Broke Interviews

#57

Post by nradov »

torben-friis wrote: post #7

I don’t understand how offline interviewing is needed to catch ai use, not counting take homes. Surely just asking the candidate to lean a bit back on the web interview and then having a regular talk without him reaching for the keyboard is enough? I guess they can have some in between layer hearing the conversation and posting tips but even then it would be obvious someone’s reading from a sheet.

That type of cat-and-mouse game is ultimately pointless. It's fairly easy to build an ambient AI assistant that will listen in to the conversation and automatically display answers to interview questions without the candidate touching a keyboard. If the interviewer wants to get any reliable signal then they'll have to ask questions that an AI can't answer effectively.

Re: AI Broke Interviews

#58

Post by captainkrtek »

ekropotin wrote: post #24
captainkrtek wrote: post #13

I’ve conducted about 60 interviews this year, and have spotted a lot of AI usage. At first I was quite concerned, then I realized that in nearly all cases I’d spotted usage, a pattern stood out. Of the folks I spotted, all spoke far too clearly and linearly when it came to problem solving. No self doubt, no suggestion of different approaches and appearance of thought, just a clear A->B solution. Then, because they of…

Jumping straight to the optimal solution may also indicate that candidate have seen the problem before.

The funny thing is, they don’t. They often jump to a solution that lacks in many ways, because it barely addresses the few inputs I gave (since they asked no follow up, even when I suggest they ask for more requirements).

Re: AI Broke Interviews

#59

Post by captainkrtek »

tavavex wrote: post #25
captainkrtek wrote: post #13

I’ve conducted about 60 interviews this year, and have spotted a lot of AI usage. At first I was quite concerned, then I realized that in nearly all cases I’d spotted usage, a pattern stood out. Of the folks I spotted, all spoke far too clearly and linearly when it came to problem solving. No self doubt, no suggestion of different approaches and appearance of thought, just a clear A->B solution. Then, because they of…

Can I ask - out of the 60 interviews, roughly how many times did you suspect AI usage?

Probably about 10 or so.

Re: AI Broke Interviews

#60

Post by captainkrtek »

Freedom2 wrote: post #27
captainkrtek wrote: post #13

I’ve conducted about 60 interviews this year, and have spotted a lot of AI usage. At first I was quite concerned, then I realized that in nearly all cases I’d spotted usage, a pattern stood out. Of the folks I spotted, all spoke far too clearly and linearly when it came to problem solving. No self doubt, no suggestion of different approaches and appearance of thought, just a clear A->B solution. Then, because they of…

> most engineers contemplate ideas, change their mind, ask clarifying questions I don't disagree at all. I find it slightly funny that in my experience interviewing for FAANG and YC startups, the signs you mentioned would be seen as "red flags". And that's not just my assumption, when I asked for feedback on the interview, I have multiple times received feedback along the lines of "candidate showed hesitation and ind…

Yeah that is definitely something that is subject to the interviewers opinion and maybe company culture. To me, question asking is a great thing, though the candidate eventually needs to start solving.
Post reply on HN