Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

coinbase.com

Post reply on HN
Print view View original

61–70 of 71 posts

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#61

Post by pards »

serial_dev wrote: post #33
pards wrote: post #28

> When the cost of building goes to zero Where are they getting all those free tokens from?!? I don't like this rhetoric. It still costs money to write code, only now that wealth gets transferred to Anthropic instead of to individual contributors.

And even ignoring the token costs, that is just not true. Sure, for some cases, the cost is going down. I and others on my team "one-shotted" impressive features in an hour of agent work that would have taken humans probably a month if done completely manually. I've also seen agents going in circles for an hour on a fix that would have taken even a junior 3 minutes to get right (after it was struggling for 10 minutes…

> orgs still just aren't really shipping that much more quality features

Writing code was never the bottleneck at large enterprises.

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#62

Post by bonoboTP »

sarmadgulzar wrote: post #59
bonoboTP wrote: post #56

Earlier quoted context omitted.

Yes, I don't know how a junior brain would read it. I already have a map of roughly what kinds of things exist out there, even if my picture is blurry, so when I read a suggestion from the AI, I have a rough ballpark of what it means, and can ask followups and triage what is promising. If you never developed this, probably as a junior you have very little to base such decisions on.

Exactly, I absolutely agree that AI supercharges seniors with judgment and conviction. They’re no longer constrained by how fast they can type. But this is about what comes next i.e. seniors of tomorrow If AI gets so good at software that vibe coding is the new norm and as Elon Musk says that AI will generate the machine code directly without any intermediate compilation or interpretation then we wouldn’t need senior…

I doubt AI will ever generate straight up machine code directly as a best practice. Hierarchical, high-level abstractions help even AI reasoning. Re-reading some undifferentiated machine code make it harder to reason about even for the AI, while having a compact representation expressing the high level algo is much more efficient.

Regarding seniors of tomorrow, I think there are big differences between good devs and sub-mediocre ones, and I think their proportion won't change, and the good ones will have the drive to use the AI to learn and understand because they are simply curious and want to know. But most programmers don't care at all, and that's the case even today, and they will be less forced to learn. But I don't think we will lose much with this. Important things are held up by a small proportion of engineers, a Pareto-like principle.

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#63

Post by bonoboTP »

dan_q wrote: post #17
bravetraveler wrote: post #15

Earlier quoted context omitted.

Fourth crack commandment: never get high on your own supply

[flagged]

They extensively use Claude to develop Claude. It's absurd to suggest they don't use what they push on their users. It's one of the most common criticisms of Claude Code that it's too vibe coded and buggy due to that.

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#64

Post by bonoboTP »

pards wrote: post #61
serial_dev wrote: post #33

Earlier quoted context omitted.

And even ignoring the token costs, that is just not true. Sure, for some cases, the cost is going down. I and others on my team "one-shotted" impressive features in an hour of agent work that would have taken humans probably a month if done completely manually. I've also seen agents going in circles for an hour on a fix that would have taken even a junior 3 minutes to get right (after it was struggling for 10 minutes…

> orgs still just aren't really shipping that much more quality features Writing code was never the bottleneck at large enterprises.

Nor was product quality the goal. Principal agent problems abound. The large org's behavior follows the thousandfold incentive gradients of the individual employees in their own career, CV building, promotion paths, budget allocation, etc.

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#65

Post by dgellow »

mohsen1 wrote: post #19
dgellow wrote: post #6

Something I appreciated when going through Anthropic interview process was that I only dealt with humans. They could have been showcasing how even their interviews are done with Claude or something, but no, the whole loop was interacting with real humans, and nothing was on the topic “how do you work with LLMs”. It was about how me, the human, think and approach situations, and how I handle working with others (human…

Hmmm... I had a different experience. They had a fully automated environment where you ha to write test that passes some tests. No human involved. And the time requirement was insanely tight

Interesting, I'm wondering if that varies depending on the hiring team? Or maybe my experience wasn't standard as it was part of an acquisition process, though I'm pretty sure I did all the steps of their interview process. Or we were in different tracks. I don't know ¯\_(ツ)_/¯

I did find my interviews (with humans) to be quite tight too

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#66

Post by vanuatu »

Muromec wrote: post #12
mbil wrote: post #10

Earlier quoted context omitted.

I’ve been conducting AI Coding interviews at my job. Candidate will screen share and use AI-assisted coding environment and tools of their choice. I ask them to implement xyz thing. What I’m looking for is how they interact with the AI agent. Do they ask the agent to plan first? Do they review the plan? Etc. It’s pretty typical stuff that you might expect an experienced engineer to do if they effectively use such too…

>It’s pretty typical stuff that you might expect an experienced engineer to do if they effectively use such tools daily. It's also the kind of stuff that somebody can learn in a week, so not hiring the right person who just didn't spend this week of time yet for whatever reason is a loss.

I used to think this was true until my company adopted a similar process, and the amount of very senior candidates who used it very ineffectively was highly surprising (eg staff candidate only using the default Google search AI), whereas the more junior candidates figured out you could one shot the problem but were unable to explain their code

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#67

Post by Muromec »

pards wrote: post #34
Muromec wrote: post #11

Earlier quoted context omitted.

I was on the interviewer side last month. We stopped giving take-home assignments because they don't produce a signal anymore. We explicitly said -- feel free to use whatever framework and AI assistant, just show us how you do it. The practical part had no leetcode too. Just build something really basic, then explain a snippet of code (3 lines) and generalize it. A trick question (with a disclosure it's a trick quest…

> people who can't do 2+2 and ones that can't understand the question without rephrasing it three times We resorted to filtering candidates in-person with 5 basic technical questions on pen and paper. And I mean _really_ basic questions. This was surprisingly effective because it filters for many non-technical skills like being able to read and write English, follow instructions, and show up to the office, on time, a…

We don't go that far. From the small sample I saw, I noticed that people who can't answer the basic questions are ones that can't use AI effectively either and don't understand that interview is for them to showcase their skills, with AI or without.

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#68

Post by Muromec »

dan_q wrote: post #21
Muromec wrote: post #16

Earlier quoted context omitted.

It signals you don't have enough time or will to chase the latest stuff, otherwise you would have learned it with other cool kids half a year earlier. It carries some signal, but I would not choose based on this alone. I would even consider selecting against this to a certain point, but I also work in a place that COBOL on a mainframe. YMMV.

> I would even consider selecting against this to a certain point I agree. A technology professional who views themselves as "one of the cool kids" (read: easily manipulated by social media) is a legitimate security threat, as are many of the popularly promoted approaches to "LLM-assisted software developement".

I haven't considered looking at it from the security thread angle. Can you expand on this?

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#69

Post by ryanSrich »

Levitz wrote: post #54
ryanSrich wrote: post #30

Earlier quoted context omitted.

> This all sounds good on paper but I have an honest question: How do you develop judgement as a junior or mid level engineer to critique AI if you yourself never learned that skill by making mistakes in the first place? Shouldn't this be a pretty fundamental part of a degree? If you're in school right now for a CS or SWE related degree, I would imagine you're learning how code actually works, how the math actually w…

These are two completely different types of mistakes. LLMs are, by now, pretty good at not making "code doesn't work" and "math doesn't work" kind of mistakes when writing code. These are also "easy" things to get good at, you learn what each part does, understand the abstractions, ensure it makes sense, and go on with your day. Unit testing helps here. LLMs are not that good at not making "this works wrong" kind of…

> but maybe it's 2 lines of code with 6 lines of comments (looking at you Claude), maybe it defines three helper functions it doesn't really need, maybe it does something "here" when it should be doing that something "there" instead, maybe it finds itself in a framework and completely disregards how the framework is supposed to do things, etc.

I would fully expect a brand new college grad to call these things out. There's nothing in here that requires extensive experience to understand. These are basic principles any SWE should know. Nonsense comments are common sense to pull, if the LLM is pulling in a framework you should look into that framework and understand how it works. Don't know the framework during the interview? Say that. Tell the person interviewing you "looks like it's pulling in XYZ. I'm not entirely familiar with that. I understand at a high level what it's doing, but I'd want to dig deeper and understand if the LLM is doing this part right"

Re: Interviewing Engineers in the AI Era: Lessons from a Year of Rebuilding

#70

Post by wiksorr »

serial_dev wrote: post #33
pards wrote: post #28

> When the cost of building goes to zero Where are they getting all those free tokens from?!? I don't like this rhetoric. It still costs money to write code, only now that wealth gets transferred to Anthropic instead of to individual contributors.

And even ignoring the token costs, that is just not true. Sure, for some cases, the cost is going down. I and others on my team "one-shotted" impressive features in an hour of agent work that would have taken humans probably a month if done completely manually. I've also seen agents going in circles for an hour on a fix that would have taken even a junior 3 minutes to get right (after it was struggling for 10 minutes…

[dead]
Post reply on HN