It's funny how this article seems to repeat itself halfway through, like it was written by AI
AI Broke Interviews
11–20 of 132 posts
Re: AI Broke Interviews
#12While I agree LLMs have forever changed the interviewing game, I also strongly disagree with deeming slop code as "perfect" and "optimal". There's a lot of shitty code made my LLMs, even today. So maybe we should lean in, and get people to critique generated code with the interviewer. Besides, being able to talk through, review, and discuss code is more important than the initial creation.
It's an indictment of how bad coding interviews are/were
Re: AI Broke Interviews
#13I’ve conducted about 60 interviews this year, and have spotted a lot of AI usage.
At first I was quite concerned, then I realized that in nearly all cases I’d spotted usage, a pattern stood out.
Of the folks I spotted, all spoke far too clearly and linearly when it came to problem solving. No self doubt, no suggestion of different approaches and appearance of thought, just a clear A->B solution. Then, because they often didn’t ask any requirements questions beyond what I initially asked, the solution would be inadequate.
The opinion I came to is that even in the best Pre-AI era interviews I conducted, most engineers contemplate ideas, change their mind, ask clarifying questions. Folks mindlessly using AI don’t do this and instead just treat me as the prompt input and repeat it back. Regardless of if they were using AI or not, I won’t know ultimately, they still fail to meet my bar.
Sure, some more clever folks will mix or limit their LLM usage and get past me, but oh well.
Re: AI Broke Interviews
#14Interviews should be in-person.
Re: AI Broke Interviews
#15I am teaching a coding class, and we had to switch to in person interview/viva assessment about the code written by students, to deal with AI written code. It works, but it requires a lot of extra effort on our side. I don't know if it is sustainable...
Why wouldn't something like this work? 1. Get students to work on a more complex than usual project (in relation to their previous peers). Let them use whatever they want and let them know that AI is fine. 2. Make them come in for a physical exam where they have questions about they why of decisions they had to take during the project. And that's it? I believe that if you can a) produce a fully working project meetin…
If I read your suggestion correctly, you're saying the exam is basically a board explaining their decision making around their code. That sounds great in theory but in practice it would be very hard to grade. Or at least, how could someone fail? If you let them use AI you can't really fault them for not understanding the code, can you? Unless you teach the course to 1. use AI and then 2. verify. And step 2 requires an understanding of coding and experience to recognize bad architecture. Which requires you to think through a problem without the AI telling you the answer.
Re: AI Broke Interviews
#16I am teaching a coding class, and we had to switch to in person interview/viva assessment about the code written by students, to deal with AI written code. It works, but it requires a lot of extra effort on our side. I don't know if it is sustainable...
Why wouldn't something like this work? 1. Get students to work on a more complex than usual project (in relation to their previous peers). Let them use whatever they want and let them know that AI is fine. 2. Make them come in for a physical exam where they have questions about they why of decisions they had to take during the project. And that's it? I believe that if you can a) produce a fully working project meetin…
A project doesn't quite work for my course, as we teaching different techniques and would like knowledge of each of them.
But yes we currently allow students to use AI provided their solution works and they can explain it. We just discourage to use AI to generate the full solution to each problem.
Re: AI Broke Interviews
#17I interviewed a guy a couple of months ago that had perfect responses to every tech question I threw at him. He even did really well on the white boarding session. The only thing was he would wait for 10-20 seconds to respond to everything. Not long enough to get called out but just long enough to notice. He aced everything. He’s a horrible employee, a senior that doesn’t seem to know anything. I almost suggested he start using his interview LLM when regular folks were asking him questions.
Re: AI Broke Interviews
#18Interviews should be in-person.
In every practical sense, online interviews are the part of the early screening process. The sheer amount of applicants means that you need to do some filtering before inviting people to do on-site interviews.
Re: AI Broke Interviews
#19The article implies that somewhat, before AI the leetcode/brainteaser/behavioral interview process had somewhat acceptable results.
The reality is that AI just blew up something that was a pile of garbage, and the result is exactly what you'd expect.
We all treat interviews in this industry as a human resources problem, when in reality is an engineering problem.
The people with the skills to assess technical competency are even more scarce than actual engineers (b/c they would be engineers with people skills for interviewing), and that kind of people is usually very very busy to be bothered with what's a (again, perceived) human resources problem.
Then the rest is just random HR personnel pretending that they know what they're talking about. AI just exposed (even more) how incompetent they are.
Re: AI Broke Interviews
#20I’ve conducted about 60 interviews this year, and have spotted a lot of AI usage. At first I was quite concerned, then I realized that in nearly all cases I’d spotted usage, a pattern stood out. Of the folks I spotted, all spoke far too clearly and linearly when it came to problem solving. No self doubt, no suggestion of different approaches and appearance of thought, just a clear A->B solution. Then, because they of…
I interviewed a guy in person and he paused for 5 seconds, then wrote a perfect solution. I tried making the problem more and more complicated and he nailed it anyway, also after a brief pause. We were done in half the time.
Maybe he just memorized the solution, I don’t know.
Would you fail that guy?