Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

coinbase.com

Post reply on HN
Print view View original

11–20 of 71 posts

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#11

Post by Muromec »

o0ower0o wrote: post #7

I am really looking forward to the day where AI is normalized during interviews. Right now the duality of interviews is basically: - You get a leetcode question and if you're lucky is an easy medium that you can solve, if you're really lucky you already solved it and can pretend you are approaching the problem the first time. Good luck if you get a hard question and you never saw it before. - You get a home assignmen…

I was on the interviewer side last month. We stopped giving take-home assignments because they don't produce a signal anymore.

We explicitly said -- feel free to use whatever framework and AI assistant, just show us how you do it. The practical part had no leetcode too. Just build something really basic, then explain a snippet of code (3 lines) and generalize it. A trick question (with a disclosure it's a trick question) if the candidate did it fast enough that we didn't have to go into the overtime. A bit of theory about protocols, all in all an hour and we leave another 30 minutes on top to answer questions.

At the end of the day we just filter out with confidently bad takes, people who can't do 2+2 and ones that can't understand the question without rephrasing it three times.

The most bizzarre candidate didn't know anything at all, but was so relaxed and confident, that he spent all of the 30 minutes asking about the company and how his day would like and all that, while he clearly bombed it.

The difficult part is how to not filter out a competent person who doesn't necessarily agree with all of your takes, uses all the same tools and had all of the same experiences as both of the interviewers.

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#12

Post by Muromec »

mbil wrote: post #10
mchinen wrote: post #5

I've been considering different ways to measure code understanding and fluency. I'm still using traditional 'easy' problems on a coding pad to probe this, but I'm finding a new generation candidates that don't prepare for this, and may be able to have a full career with just a higher level of code understanding. But they might not ever develop the ride-a-bike with one hand level of fluency unless they grind pointless…

I’ve been conducting AI Coding interviews at my job. Candidate will screen share and use AI-assisted coding environment and tools of their choice. I ask them to implement xyz thing. What I’m looking for is how they interact with the AI agent. Do they ask the agent to plan first? Do they review the plan? Etc. It’s pretty typical stuff that you might expect an experienced engineer to do if they effectively use such too…

>It’s pretty typical stuff that you might expect an experienced engineer to do if they effectively use such tools daily.

It's also the kind of stuff that somebody can learn in a week, so not hiring the right person who just didn't spend this week of time yet for whatever reason is a loss.

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#13

Post by pmg101 »

Muromec wrote: post #12
mbil wrote: post #10

Earlier quoted context omitted.

I’ve been conducting AI Coding interviews at my job. Candidate will screen share and use AI-assisted coding environment and tools of their choice. I ask them to implement xyz thing. What I’m looking for is how they interact with the AI agent. Do they ask the agent to plan first? Do they review the plan? Etc. It’s pretty typical stuff that you might expect an experienced engineer to do if they effectively use such too…

>It’s pretty typical stuff that you might expect an experienced engineer to do if they effectively use such tools daily. It's also the kind of stuff that somebody can learn in a week, so not hiring the right person who just didn't spend this week of time yet for whatever reason is a loss.

Yeah it tells you nothing of value. It's the Recency Bias taken to its ultimate conclusion isn't it. Anything I learned about AI I learned in the last few months but somehow that's as important as all the experience of my decades long software career? Come on.

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#15

Post by bravetraveler »

dgellow wrote: post #6

Something I appreciated when going through Anthropic interview process was that I only dealt with humans. They could have been showcasing how even their interviews are done with Claude or something, but no, the whole loop was interacting with real humans, and nothing was on the topic “how do you work with LLMs”. It was about how me, the human, think and approach situations, and how I handle working with others (human…

Fourth crack commandment: never get high on your own supply

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#16

Post by Muromec »

pmg101 wrote: post #13
Muromec wrote: post #12

Earlier quoted context omitted.

>It’s pretty typical stuff that you might expect an experienced engineer to do if they effectively use such tools daily. It's also the kind of stuff that somebody can learn in a week, so not hiring the right person who just didn't spend this week of time yet for whatever reason is a loss.

Yeah it tells you nothing of value. It's the Recency Bias taken to its ultimate conclusion isn't it. Anything I learned about AI I learned in the last few months but somehow that's as important as all the experience of my decades long software career? Come on.

It signals you don't have enough time or will to chase the latest stuff, otherwise you would have learned it with other cool kids half a year earlier. It carries some signal, but I would not choose based on this alone. I would even consider selecting against this to a certain point, but I also work in a place that COBOL on a mainframe. YMMV.

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#17

Post by dan_q »

bravetraveler wrote: post #15
dgellow wrote: post #6

Something I appreciated when going through Anthropic interview process was that I only dealt with humans. They could have been showcasing how even their interviews are done with Claude or something, but no, the whole loop was interacting with real humans, and nothing was on the topic “how do you work with LLMs”. It was about how me, the human, think and approach situations, and how I handle working with others (human…

Fourth crack commandment: never get high on your own supply

[flagged]

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#18

Post by keybored »

dgellow wrote: post #6

Something I appreciated when going through Anthropic interview process was that I only dealt with humans. They could have been showcasing how even their interviews are done with Claude or something, but no, the whole loop was interacting with real humans, and nothing was on the topic “how do you work with LLMs”. It was about how me, the human, think and approach situations, and how I handle working with others (human…

AI critics and honest AI executives of 2023 might say that AI companies are automating labor so that Capital can capture all productive value. But we have to give them some credit. They are hiring real humans. And not even hounding them about how they are not being sufficiently useless-meatbag relay points for the AI models and agents. And that demonstrates that they are actually being real humane mensch.

Meanwhile, how many jobs have AI critics created? Crickets.

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#19

Post by mohsen1 »

dgellow wrote: post #6

Something I appreciated when going through Anthropic interview process was that I only dealt with humans. They could have been showcasing how even their interviews are done with Claude or something, but no, the whole loop was interacting with real humans, and nothing was on the topic “how do you work with LLMs”. It was about how me, the human, think and approach situations, and how I handle working with others (human…

Hmmm... I had a different experience. They had a fully automated environment where you ha to write test that passes some tests. No human involved. And the time requirement was insanely tight

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#20

Post by normie3000 »

bravetraveler wrote: post #15
dgellow wrote: post #6

Something I appreciated when going through Anthropic interview process was that I only dealt with humans. They could have been showcasing how even their interviews are done with Claude or something, but no, the whole loop was interacting with real humans, and nothing was on the topic “how do you work with LLMs”. It was about how me, the human, think and approach situations, and how I handle working with others (human…

Fourth crack commandment: never get high on your own supply

It's why you never meet a dog that works in a dogfood factory.
Post reply on HN