Live data from Hacker News

Productivity gains from AI coding assistants haven’t budged past 10% – survey

shiftmag.dev

51–60 of 99 posts

Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey

#51

Earlier quoted context omitted.

There won't ever be newer training data. The OG data came from sites like Stackoverflow. These sites will stop existing once LLMs become better and easier to use. Game over.

Every time claude code runs tests or builds after a change, it's collecting training data.

You need human language programming-related questions to train on too, not just the code.

Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey

#52
post #31
post #5

You're only as fast as your biggest bottleneck. Adding AI to an existing organization is just going to show you where your bottlenecks are, it's not going to magically make them go away. For most companies, the speed of writing code probably wasn't the bottleneck in the first place.

Agreed. The bottleneck is QA/Code review and that is never going away from most corps. I've never worked at a job in tech that didn't require code review and no, asking a code agent to review a PR is never going to be "good enough". And here we are, the central argument for why code agents are not these job killing hype beasts that are so regularly claimed. Has anyone seen what multi-agent code workflows produce? Tak…

i dont think thats actually the bottleneck?

the bottleneck is aligning people on what the right thing to do is, and fiting the change into everyone's mental models. it gets worse the more people are involved

Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey

#53
post #50

A 10% uplift in productivity for the cost of probably 0.001% of the salary budget is an incredible success.

This is exactly right. And assuming organizations use the gains to cut headcount rather than boost total productivity, a 10% reduction in white collar employment would still be an era-defining systemic shock to the economy.

Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey

#54
post #26

Apparently "AI is speeding up the onboarding process", they say. But isn't that because the onboarding process is about learning, and by having an AI regurgitate the answers you can complete the process without learning anything, which might speed it up but completely defeats the purpose?

I think there's definite scope for that being true; not because you can start doing stuff before you understand it (you can), but because you can ask questions of a codebase your unfamiliar with to learn about it faster.

Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey

#55

Earlier quoted context omitted.

No to the first question, and maybe with a lot of money for the second question.

In the 20 years I've been in the industry, boiler plate has dropped dramatically in the backend. Right now, front end has tons of boiler plate. It's one of the reasons AI hassle such a wow factor for FE, trivial tasks require a lot of code. But even that is much better than it was 10 years ago. That was a long way of saying I disagree with your no.

FE has a lot of boilerplate only if you’re starting from scratch every single time. That’s why we had template systems and why we invented view libraries. Once you’ve defined your libraries, you just copy-paste stuff.

Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey

#56

Earlier quoted context omitted.

> let's have LLMs check our code for correctness Lmao. Rofl even. (Testing is the one thing you would never outsource to AI.)

Outsourcing testing to AI makes perfect sense if you assume that tests exist out of an obligation to meet some code coverage requirements, rather than to ensure correctness. Often I'll write a module and a few tests that cover its functionality, only for CI to complain that line coverage has decreased and reject my merge! AI to the rescue! A perfect job for a bullshit generator.

outsourcing testing the AI also gets its code to be connected to deterministic results, and show let the agent interact with the code to speculate expectations and check them against the actual code.

it could still speculate wrong things, but it wont speculate that the code is supposed to crash on the first line of code

Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey

#57

This is self-reported productivity, in that devs are saying AI saves them about 4 hours per week. But let’s not forget the METR study that found a 20% increase in self-reported productivity but a 19% decrease in actual measured productivity. (It used a clever and rigorous technique for measuring productivity differences, BTW, for anyone as skeptical of productivity measures as I am.)

Has the METR study been replicated?

Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey

#58
post #14
post #5

You're only as fast as your biggest bottleneck. Adding AI to an existing organization is just going to show you where your bottlenecks are, it's not going to magically make them go away. For most companies, the speed of writing code probably wasn't the bottleneck in the first place.

the amount of people that work in technology and have never heard of amdahl's law always shocks me https://en.wikipedia.org/wiki/Amdahl's_law a 100% increase in coding speed means I then I get to spend an extra 30 minutes a week in meetings while now hating my job, because the only fun bit has been removed "progress"

So if I'm understanding you correctly, prior to AI tools you spent 1 hour per week coding? And now you spend 30 minutes per week?

Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey

#59

Earlier quoted context omitted.

Every time claude code runs tests or builds after a change, it's collecting training data.

You need human language programming-related questions to train on too, not just the code.

thats what the related chats are for?

Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey

#60

This is self-reported productivity, in that devs are saying AI saves them about 4 hours per week. But let’s not forget the METR study that found a 20% increase in self-reported productivity but a 19% decrease in actual measured productivity. (It used a clever and rigorous technique for measuring productivity differences, BTW, for anyone as skeptical of productivity measures as I am.)

https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...

That info is from mid 2025, talking about models released in Oct 2024 and Feb 2025. It predates tools like Claude Code and Codex, Lovable was 1/3 current ARR, etc.

This might still be true but we desperately need new data.

Post reply on HN