I’ve conducted about 60 interviews this year, and have spotted a lot of AI usage. At first I was quite concerned, then I realized that in nearly all cases I’d spotted usage, a pattern stood out. Of the folks I spotted, all spoke far too clearly and linearly when it came to problem solving. No self doubt, no suggestion of different approaches and appearance of thought, just a clear A->B solution. Then, because they of…
I interviewed a guy in person and he paused for 5 seconds, then wrote a perfect solution. I tried making the problem more and more complicated and he nailed it anyway, also after a brief pause. We were done in half the time. Maybe he just memorized the solution, I don’t know. Would you fail that guy?
In those cases where I’ve seen that level of performance, there have been (one or more of):
- Audio/video glitches.
- candidate pausing frequently after each question, no words, then sudden clarity and fluency on the problem.
- candidate often suggests multiple specific ideas/points to each question I ask.
- I can often see their eyes reading back and forth (note; if you use AI in an interview, maybe dont use a 4K webcam).
- way too much specificity when I didn’t ask for it. For example, the topic of profiling a go application came up, and the candidate suggested we use go tool pprof and suggested a few specific arguments that weren’t relevant, later I found in the documentation the same exact example commands verbatim.
In all, the impression I come away with in those types of interviews is that they performed “too well” in an uncanny way.
I worked for AWS for a long time and did a couple hundred interviews there, the best candidates I interviewed were distinctly different in how they solved problems, how they communicated, in ways that reading from an llm response can’t resemble.