Live data from Hacker News

Interviewing a software engineer who prepared with AI

kapwing.com

11–20 of 731 posts

Re: Interviewing a software engineer who prepared with AI

#11
Potentially important side points, since not everyone knows, and we don't want anyone to learn a mistake by example:

1. Don't use blur to redact documents. Whatever blur was used can probably be reversed.

2. Don't try to hide the identity of someone you're talking about by redacting a few details on their resume. With the prevalence of public and private resume databases, that's probably easy to match up with a name.

Re: Interviewing a software engineer who prepared with AI

#12

I recently interviewed an engineer who was somehow using ChatGPT realtime on another laptop beside him. The irony was that the questions were pretty simple overall and our rubric also wasn't very strict, so he likely would have passed if he just used his memory and common sense. Though the answers weren't wrong overall, I still felt cheated because of the deception and had to reject him later.

I rejected an applicant earlier this year who was obviously reading off another screen for every answer (as in, blatant pauses while I could literally see their eyes moving back and forth). I don't understand if they thought I wouldn't notice or what.

Re: Interviewing a software engineer who prepared with AI

#13
post #4

It says "Prepared with AI" in the title, but the article is about someone who blatantly lied about their past experience in the interview. The AI was used as a tool to generate false stories, but that's not what I assumed when I read the title. It's common for people to "prepare" with LLMs by having them review resumes and suggest changes, but asking an LLM to wholesale fabricate things for you is something else enti…

I had interpreted it as some of the answers being given during the interview had been generated by an LLM, which then choked when it was met with a more sophisticated query of how several of the answers connected together. Was this not the case?

Re: Interviewing a software engineer who prepared with AI

#14
> I ended by saying that the software community is smaller than it seems, and integrity and reputation goes a long way.

Well who are they? How would the next member of the community know this is a fake candidate. I like the idea in general of finding a way to eliminate these time-wasters but how would that work? The candidate can adjust a bit and improve the AI "foo" to come up with online answers for them.

Re: Interviewing a software engineer who prepared with AI

#15
Oddly one impact on me from reading this is that Kapwing seems like probably a nice place to apply for a job -- simple enough application process, human review, sane and respectful take-home and no live pressure coding. I'm not affiliated in any way nor am I a FT software developer, but this seemed like a pretty sane process (which sadly the article reveals may not be sufficient to properly vet candidates).

Re: Interviewing a software engineer who prepared with AI

#16
Had an interesting live coding screen where the candidate was coding a solution, dropped from the call and screenshare for 20 minutes, showed back up with a full solution different from what they had before dropping and carried on as if nothing happened.

Re: Interviewing a software engineer who prepared with AI

#17
post #2

An interesting story! I've also had an AI cheater during phone screen, but they were pretty clumsy... A question of form "You mentioned you used TechX on your resume, tell me more what you did with it" was answered with a long-winded but generic description of TechX and zero information about their project or personal contribution. Another thing that I can take away from that is "take home project" is no longer a goo…

I've had situations where I submitted a take-home exercise, only for me to get feedback that it didn't match their required level.

After some back & forth I was able to (politely) prove their feedback was not correct, which actually granted me a follow-up interview.

Unfortunately, this was a unicorn, most companies don't give feedback, let alone admit they were wrong.

But, take-home is preferred, I want to use my IDE, with my keyboard shortcuts etc.

Then there are take-home timed challenges on systems like hackerrank / leetcode etc, which are horrible in terms of accessibility and access. Not to mention that they are a pass/fail, and focus purely on speed, not quality.

Next to that they don't allow you to work in an environment you're comfortable in. No debugger, etc. When an HVAC company hires a new tech, do they tell him/her to do a 1.5 hour repair with only a hammer and a lighter to diagnose and fix an issue? No, it's stupid. Why do developers have to do this then?

And the same applies to live coding exercises. While there is an opportunity to explain yourself, you're still in an extremely uncomfortable environment. Why is there such an emphasis to put people in an environment where they are not set up to succeed?

Re: Interviewing a software engineer who prepared with AI

#18
post #2

An interesting story! I've also had an AI cheater during phone screen, but they were pretty clumsy... A question of form "You mentioned you used TechX on your resume, tell me more what you did with it" was answered with a long-winded but generic description of TechX and zero information about their project or personal contribution. Another thing that I can take away from that is "take home project" is no longer a goo…

Take home projects >>>>> live coding sessions, unless you're interviewing for some kind of twitch streamer position.

Just have a 1 hour or 2 hour call with candidate where you guys go through the project.

Re: Interviewing a software engineer who prepared with AI

#19

Had an interesting live coding screen where the candidate was coding a solution, dropped from the call and screenshare for 20 minutes, showed back up with a full solution different from what they had before dropping and carried on as if nothing happened.

[dead]
Post reply on HN