Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Interviewing a software engineer who prepared with AI

kapwing.com

Post reply on HN
Print view View original

151–160 of 731 posts

Re: Interviewing a software engineer who prepared with AI

#151

Post by roguecoder »

Ylpertnodi wrote: post #125
gwbas1c wrote: post #112

Some people think it's perfectly normal to stretch the truth on a resume, and to lie in an interview. Other people think an interview is just a matter of finding the "magic words" to get the job. What I don't understand is, what did the candidate do with AI? Did they use the AI as a coach? Did they use it to suggest edits to the resume? --- I once interviewed a candidate who was given my questions in advance. (I shou…

>I couldn't just make up new questions for every candidate. Ask each candidate the same questions?

Follow up questions will vary, but the bulk of most interviews is the same for every candidate, and candidates are then judged based on a rubric that is the same for every candidate (though often tailored to the specific role).

The consistency lets interviewers compare across candidates, and avoids the cognitive pitfall of defining a rubric after-the-fact that lets us hire the candidate who appealed to our lizard brains.

Even at startups, questions are also usually tested on several existing employees before it is used on the first external candidate, for calibration. Companies put a lot of time and money trying to hire for actual competence.

Re: Interviewing a software engineer who prepared with AI

#152

Post by robocat »

> For prospective job candidates, my advice is still that "the truth will set you free"

Is that really good advice?

If you have the wisdom of knowing when to embellish and when to blur, then you're more likely to get a job and more likely to fit in.

I'm a spectrum, and generally I'm over-truthful and I notice my habit regularly affects me negatively.

Re: Interviewing a software engineer who prepared with AI

#154

Post by Clubber »

>The next stage of our interview process, had this candidate moved forward, is to implement a take-home project that we have specifically designed for prospective candidates to complete. Had we moved this candidate forward, I have no doubt that they would have been able to use AI to pass the take home project with flying colors.

I have no doubt as well, but I couldn't help but noticing, "Don't bother with take home tests," wasn't on the list of remedies.

Re: Interviewing a software engineer who prepared with AI

#155

Post by heelix »

crooked-v wrote: post #12
lurker919 wrote: post #10

I recently interviewed an engineer who was somehow using ChatGPT realtime on another laptop beside him. The irony was that the questions were pretty simple overall and our rubric also wasn't very strict, so he likely would have passed if he just used his memory and common sense. Though the answers weren't wrong overall, I still felt cheated because of the deception and had to reject him later.

I rejected an applicant earlier this year who was obviously reading off another screen for every answer (as in, blatant pauses while I could literally see their eyes moving back and forth). I don't understand if they thought I wouldn't notice or what.

I've had some fun ones, when interviewing folks - back in the days when people where hiring.

* had someone make a giant cheat sheet with interview questions and taped it to the wall behind their computer. Part way through, the tape gave out and covered him.

* had someone attempt to lip sync the answers. The guy talking and the guy on screen were not the same guy. There was a bit of pretend 'oh just lag' for a while.

* Person we interviewed was not the same one who showed up for work. Great answers, great experience on the interview. Asked about some things we had talked about for quite a while - and he could not recall anything. Came to realize not the same person.

* the glorious mechanical keyboard furiously googling for an answer.

* the sample project they were asked to create as starting point for the interview, they had never run before. They sat and read through what was likely AI generated docs to run the app. Took them a while to realize they needed something other than Java 8 installed to run the sample.

Re: Interviewing a software engineer who prepared with AI

#156

Post by gwbas1c »

ghaff wrote: post #139
gwbas1c wrote: post #112

Some people think it's perfectly normal to stretch the truth on a resume, and to lie in an interview. Other people think an interview is just a matter of finding the "magic words" to get the job. What I don't understand is, what did the candidate do with AI? Did they use the AI as a coach? Did they use it to suggest edits to the resume? --- I once interviewed a candidate who was given my questions in advance. (I shou…

The first part sounds like what I’d expect a serious candidate to do. Didn’t you look at the questions we sent you? The second part sounds like areal curveball unless you made it clear that the questions sent out were only representative/samples of what you’d ask.

The candidate wasn't supposed to know the questions in advance.

> The second part sounds like areal curveball

That was the point. The candidate wasn't supposed to know the questions in advance. Once the candidate can practice / memorize, there's no way to evaluate the candidate.

Re: Interviewing a software engineer who prepared with AI

#157

Post by roguecoder »

hbsbsbsndk wrote: post #61

I've interviewed some candidates (more senior than TFA) and I agree with OP that it is a uniquely uncomfortable experience. Candidates who rely on AI seem to just be totally turning their brains off. At least a candidate who was embellishing in the old days would try to BS if they were caught. They could try and fill in the blanks. These candidates give plausible-sounding answers and then truly just give up and say "…

I suspect we are seeing the first wave of programmers who got a promotion to "senior" on the basis of being an early AI adopter at a place that valued lines of code written or tickets closed or other similarly-game-able metrics.

Alternatively, there are people who haven't been promoted but think their AI-fu is so good they obviously should have been, without realizing that "senior" is actually a different role, with additional responsibilities.

I've found asking about their pedagogy when coaching junior engineers is a great sorting strategy right now. It isn't something a lot of people have written about so ChatGPT's answers are full of useless platitudes, and mid-level engineers often don't even know that it is part of the job.

Re: Interviewing a software engineer who prepared with AI

#158

Post by drivingmenuts »

jaredklewis wrote: post #110
no-dr-onboard wrote: post #45

I keep coming back to this phrase used in this post: "it was scary". Yeah, hiring is scary. Hiring is insanely expensive on all fronts. Firing people is difficult, it's expensive and legally exposing. Hiring the wrong person, allowing them access your systems and potentially exfiltrate your IP to them is a hazardous but necessary venture. The thing is, none of these things really changed with AI. People have been lyi…

> - And yeah, you guys, it's time to buy a suit. Others have already commented on this, but do you work in tech? IME getting interviewed by directors and even VPs in t-shirts is the norm. I’ve worn jeans to work my whole career. If anything, I think people in tech have a strong prejudice against people in suits (ie “obviously this person isn’t a real software engineer, they’re wearing a suit.”) Anyway, probably not g…

The general rule seems to be if you’re not customer-facing, then no suit is needed. Just wear clean, neat clothes and that’s usually enough. If a suit or uniform is needed, that would be noted up front.

Re: Interviewing a software engineer who prepared with AI

#159

Post by roguecoder »

mathgeek wrote: post #77
madeofpalk wrote: post #48

Earlier quoted context omitted.

The twist on “I used AI” to this though is that everyone comes out looking the same. They all have the same resume format, made by the same tool, stuffed with the same keywords.

Fair for the ones who don’t put in any effort, but I don’t buy this generalization for the folks who are real people in the middle between “completely unqualified” and “telling the truth about their experience”.

Any effective screening strategy is going to catch the liars who do it only a little with some probability.

Not least because being willing to be dishonest during an interview is a strong signal the candidate will be dishonest while they are employed as well, and companies want very much to not hire those people.

Re: Interviewing a software engineer who prepared with AI

#160

Post by sterlind »

roguecoder wrote: post #133
roguecoder wrote: post #127

Earlier quoted context omitted.

You assume we aren't going to end up wearing them daily in order to get promotions. The same elitist power trip that drove RTO is also likely to produce the "dress like us & be rewarded" dynamics that push for conformity.

In more detail, in a glutted market with an unknown percentage of fakers, companies look for costly signals they can use to sort. This particular signal also indicates a willingness to set aside one's individual ego in order to assimilate in the workplace, which is especially valuable to the companies demanding developers abandon good sense in order to push AI adoption. If we want that signals to not be "a suit", it…

> The highest-paying developer jobs have always expected programmers to wear suits to the interviews.

this is the inverse of the hacker aesthetic. you might be right, but it's just sad.

personally, I'd assume the candidates that look the most non-conforming would be more talented and creative - more likely to love the work than the paycheck - but maybe it's no surprise that the highest-paying positions look for suits like quants at an investment bank.

Post reply on HN