Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI killed the tech interview. Now what?

kanenarraway.com

Post reply on HN
Print view View original

131–140 of 651 posts

Re: AI killed the tech interview. Now what?

#131

Post by ToucanLoucan »

Shorel wrote: post #106
brainwipe wrote: post #71

I was asked by an SME to code on a whiteboard for an interview (in 2005? I think?). I asked if I could have a computer, they said no. I asked if I would be using a whiteboard during my day-to-day. They said no. I asked why they used whiteboards, they said they were mimicking Google's best practice. That discussion went on for a good few minutes and by the end of it I was teetering on leaving because the fit wasn't go…

2005? You were in the right. Today? Now that's when it is tricky. How can we know you are not one of these prompt "engineers" copy paster? That's the issue being discussed. 20 years and many new technologies of difference.

What is the functional difference between copying an AI answer and copying a StackOverflow answer, in terms of it being "cheating" during an interview?

I think the entire question is missing the forest for the trees. I have never asked a candidate to write code in any fashion during an interview. I talk to them. I ask them how they would solve problems, chase down bugs, or implement new features. I ask about concepts like OOP. I ask about what they've worked on previously, what they found interesting, what they found frustrating, etc.

Languages are largely teachable, it's just syntax and keywords. What I can't teach people is how to think like programmers need to: how to break down big, hard problems into smaller problems and implement solutions. If you know that, I can teach you fucking Swift, it isn't THAT complicated and there's about 5 million examples of "how do I $X" available all over the Internet.

Re: AI killed the tech interview. Now what?

#132

Post by perlgeek »

Company A wants to hire an engineer, an AI could solve all their tech interview questions, so why not hire that AI instead?

There's very likely a real answer to that question, and that answer should shape the way that engineer should be assessed and hired.

For example, it could be that the company wants the engineer to do some kind of assessment whether a feature should be implemented at all, and if yes, in what way. Then you could, in an interview, give a bit of context and then ask the candidate to think out loud about an example feature request.

It seems to me the heart of the problem is that companies aren't very clear about what value the engineers add, and so they have trouble deciding whether a candidate could provide that value.

Re: AI killed the tech interview. Now what?

#133

Post by mclau156 »

wruza wrote: post #84
CharlieDigital wrote: post #73

Code reviews. Teams are really sleeping on code reviews as an assessment tool. As in having the candidate review code. A junior, mid, senior, staff are going to see very different things in the same codebase. Not only that, as AI generated code becomes more common, teams might want to actively select for devs that can efficiently review code for quality and correctness. I went through one interview with a YC company…

Is there a site where one could review some code and see what many others say about it and their experience level? I guess it would degrade to stackoverflow-like poems eventually, but still interesting.

github and issues page

Re: AI killed the tech interview. Now what?

#134

Post by user99999999 »

FirmwareBurner wrote: post #62
micheles wrote: post #61

Nowadays I am on the other part of the fence, I am the interviewer. We are not a FAANG, so we just use a SANE interview process. Single interview, we ask the candidate about his CV and what his expectations are, what are his competences and we ask him to show us some code he has written. That's all. The process is fast and extremely effective. You can discriminate week candidates in minutes.

>we ask him to show us some code he has written How do you expect them to get access to the property internal Git repo codebase and approval from their employer's lawyers to show it to third parties during the interview? Sounds like you're only selecting Foss devs and nothing more.

My worst code is always what I wrote yesterday. Often what’s missing is context, unless I comment ad nauseam. Sure I didn’t write complete test, obey open closed principles abstract into factory functions. The code I send from my hobby projects is likely a mess, because finishing on my own time by my own unpaid constraints wills it to be so

Re: AI killed the tech interview. Now what?

#135

Post by user99999999 »

ramon156 wrote: post #64
FirmwareBurner wrote: post #62

Earlier quoted context omitted.

>we ask him to show us some code he has written How do you expect them to get access to the property internal Git repo codebase and approval from their employer's lawyers to show it to third parties during the interview? Sounds like you're only selecting Foss devs and nothing more.

Most people have still written code for school or a hobby project. Maybe I'm missing empathy, but I cannot understand how some developers have no code to show. If that's the case however, just let them make a small project over the weekend and then do another interview where you ask stuff about what they've made. It's not that deep

Please share your GitHub @

Re: AI killed the tech interview. Now what?

#136

Post by jerf »

timr wrote: post #103
Rhapso wrote: post #96

Earlier quoted context omitted.

Yeah, very bad fit. Surprised they made an offer. Folks getting mad about whiteboard interviews is a meme at this point. It misses the point. We CANT test you effectively on your programming skillbase. So we test on a more relevant job skill, like can you have a real conversation (with a whiteboard to help) about how to solve the problem. It isn't that your interviewer knew all the languages, but that the language di…

> We CANT test you effectively on your programming skillbase. So we test on a more relevant job skill, like can you have a real conversation (with a whiteboard to help) about how to solve the problem. Except, that's not what happens. In basically every coding interview in my life, it's been a gauntlet: code this leetcode medium/hard problem while singing and tapdancing backwards. Screw up in any way -- or worse (and…

They generally do not know what they are looking for. They are generally untrained, and if they are trained, the training is probably all about using leetcode-type problems to give out interviews that are sufficiently similar that you can run stats on the results and call them "objective", which is exactly the thing we are all quite correctly complaining about. Which is perhaps anti-training.

The problem is that the business side wants to reduce it to an objective checklist, but you can't do that because of Goodhart's Law [1]. AI is throwing this problem into focus because it is basically capable of passing any objective checklist, with just a bit of human driving [2]. Interviews can not consist of "I'm going to ask a question and if you give me the objectively correct answer you get a point and if you do not give the objectively correct answer you do not". The risk of hiring someone who could give the objectively correct answers but couldn't program their way out of a wet paper bag, let alone do requirements elicitation in collaboration with other humans or architecture or risk analysis or any of the many other things that a real engineering job consists of, was already pretty high before AI.

But if interviewing is not a matter of saying the objectively correct things, a lot of people at all levels are just incapable of handling it after that. The Western philosophical mindset doesn't handle this sort of thing very well.

[1]: https://en.wikipedia.org/wiki/Goodhart%27s_law

[2]: Note this is not necessarily bad because "AI bad!", but, if all the human on the other end can offer me is that they can drive the AI, I don't need them. I can do it myself and/or hire any number of other such people. You need to bring something to the job other than the ability to drive an AI and you need to demonstrate whatever that is in the interview process. I can type what you tell me into a computer and then fail to comprehend the answer it gives is not a value-add.

Re: AI killed the tech interview. Now what?

#137

Post by aleph_minus_one »

ToucanLoucan wrote: post #131
Shorel wrote: post #106

Earlier quoted context omitted.

2005? You were in the right. Today? Now that's when it is tricky. How can we know you are not one of these prompt "engineers" copy paster? That's the issue being discussed. 20 years and many new technologies of difference.

What is the functional difference between copying an AI answer and copying a StackOverflow answer, in terms of it being "cheating" during an interview? I think the entire question is missing the forest for the trees. I have never asked a candidate to write code in any fashion during an interview. I talk to them. I ask them how they would solve problems, chase down bugs, or implement new features. I ask about concepts…

> Languages are largely teachable, it's just syntax and keywords.

This is like "learning a natural language is just 'cramming vocabulary and grammar' - voila, you've become a fluent C1 speaker". :-)

Seriously: if you argue this way, you have only seen a very biased set of programming languages, and additionally, your knowledge of these programming languages is very superficial (i.e. you have never gotten to the "interesting"/"deep" concepts that make this particular programming language special, and which are hard to replicate in most other programming languages).

Re: AI killed the tech interview. Now what?

#138

Post by michpoch »

jakubmazanec wrote: post #83
michpoch wrote: post #74

Earlier quoted context omitted.

> Also, the best (albeit the most expensive) selection process is simply letting the new person to do the actual work for a few weeks. What kind of desperate candidate would agree to that? Also, what do you expect to see from the person in a few weeks? Usual onboarding (company + project) will take like 2-3 months before a person is efficient.

Candidate would be compensated, obviously. That's why it's expensive. You don't need him to become efficient. Also I don't think it is always necessary to have such long onboarding. I'll never understand why a new hire (at least in senior position) can't start contributing after a week.

> can't start contributing after a week.

Because you have zero context of what the org is working on.

Re: AI killed the tech interview. Now what?

#139

Post by jghn »

CharlieDigital wrote: post #73

Code reviews. Teams are really sleeping on code reviews as an assessment tool. As in having the candidate review code. A junior, mid, senior, staff are going to see very different things in the same codebase. Not only that, as AI generated code becomes more common, teams might want to actively select for devs that can efficiently review code for quality and correctness. I went through one interview with a YC company…

This is harder than it sounds, although I agree in a vacuum the idea is a good one.

So much value of the code review comes from having actual knowledge of the larger context. Mundane stuff like formatting quirks and obvious bad practices should be getting hoovered up by the linters anyways. But what someone new may *not* know is that this cruft is actually important for some arcane reason. Or that it's important that this specific line be super performant and that's why stylistically it's odd.

The real failure mode I worry about here is how much of this stuff becomes second nature to people on a team. They see it as "obvious" and forgot that it's actually nuance of their specific circumstances. So then a candidate comes in and misses something "obvious", well, here's the door.

Re: AI killed the tech interview. Now what?

#140

Post by bitwizeshift »

perlgeek wrote: post #132

Company A wants to hire an engineer, an AI could solve all their tech interview questions, so why not hire that AI instead? There's very likely a real answer to that question, and that answer should shape the way that engineer should be assessed and hired. For example, it could be that the company wants the engineer to do some kind of assessment whether a feature should be implemented at all, and if yes, in what way.…

Tech interviews in general need to be overhauled, and if they were it’d be less likely that AI would be as helpful in the process to begin with (at least for LLMs in their current state).

Current LLMs can do some basic coding and stitch it together to form cool programs, but it struggles at good design work that scales. Design-focused interviews paired with soft-skill-focus is a better measure of how a dev will be in the workplace in general. Yet, most interviews are just “if you can solve this esoteric problem we don’t use at all at work, you are hired”. I’d take a bad solution with a good design over a good solution with a bad design any day, because the former is always easier to refactor and iterate on.

AI is not really good at that yet; it’s trained on a lot of public data that skews towards worse designs. It’s also not all that great at behaving like a human during code reviews; it agrees too much, is overly verbose, it hallucinates, etc.

Post reply on HN