Earlier quoted context omitted.
There won't ever be newer training data. The OG data came from sites like Stackoverflow. These sites will stop existing once LLMs become better and easier to use. Game over.
Every time claude code runs tests or builds after a change, it's collecting training data.
Productivity gains from AI coding assistants haven’t budged past 10% – survey
51–60 of 99 posts
Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey
#52You're only as fast as your biggest bottleneck. Adding AI to an existing organization is just going to show you where your bottlenecks are, it's not going to magically make them go away. For most companies, the speed of writing code probably wasn't the bottleneck in the first place.
Agreed. The bottleneck is QA/Code review and that is never going away from most corps. I've never worked at a job in tech that didn't require code review and no, asking a code agent to review a PR is never going to be "good enough". And here we are, the central argument for why code agents are not these job killing hype beasts that are so regularly claimed. Has anyone seen what multi-agent code workflows produce? Tak…
the bottleneck is aligning people on what the right thing to do is, and fiting the change into everyone's mental models. it gets worse the more people are involved
Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey
#53A 10% uplift in productivity for the cost of probably 0.001% of the salary budget is an incredible success.
Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey
#54Apparently "AI is speeding up the onboarding process", they say. But isn't that because the onboarding process is about learning, and by having an AI regurgitate the answers you can complete the process without learning anything, which might speed it up but completely defeats the purpose?
Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey
#55Earlier quoted context omitted.
No to the first question, and maybe with a lot of money for the second question.
In the 20 years I've been in the industry, boiler plate has dropped dramatically in the backend. Right now, front end has tons of boiler plate. It's one of the reasons AI hassle such a wow factor for FE, trivial tasks require a lot of code. But even that is much better than it was 10 years ago. That was a long way of saying I disagree with your no.
Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey
#56Earlier quoted context omitted.
> let's have LLMs check our code for correctness Lmao. Rofl even. (Testing is the one thing you would never outsource to AI.)
Outsourcing testing to AI makes perfect sense if you assume that tests exist out of an obligation to meet some code coverage requirements, rather than to ensure correctness. Often I'll write a module and a few tests that cover its functionality, only for CI to complain that line coverage has decreased and reject my merge! AI to the rescue! A perfect job for a bullshit generator.
it could still speculate wrong things, but it wont speculate that the code is supposed to crash on the first line of code
Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey
#57This is self-reported productivity, in that devs are saying AI saves them about 4 hours per week. But let’s not forget the METR study that found a 20% increase in self-reported productivity but a 19% decrease in actual measured productivity. (It used a clever and rigorous technique for measuring productivity differences, BTW, for anyone as skeptical of productivity measures as I am.)
Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey
#58You're only as fast as your biggest bottleneck. Adding AI to an existing organization is just going to show you where your bottlenecks are, it's not going to magically make them go away. For most companies, the speed of writing code probably wasn't the bottleneck in the first place.
the amount of people that work in technology and have never heard of amdahl's law always shocks me https://en.wikipedia.org/wiki/Amdahl's_law a 100% increase in coding speed means I then I get to spend an extra 30 minutes a week in meetings while now hating my job, because the only fun bit has been removed "progress"
Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey
#59Re: Productivity gains from AI coding assistants haven’t budged past 10% – survey
#60This is self-reported productivity, in that devs are saying AI saves them about 4 hours per week. But let’s not forget the METR study that found a 20% increase in self-reported productivity but a 19% decrease in actual measured productivity. (It used a clever and rigorous technique for measuring productivity differences, BTW, for anyone as skeptical of productivity measures as I am.)
That info is from mid 2025, talking about models released in Oct 2024 and Feb 2025. It predates tools like Claude Code and Codex, Lovable was 1/3 current ARR, etc.
This might still be true but we desperately need new data.