Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The gauge broke: devs felt 20% faster with AI, measured 19% slower (2025)

intrepidkarthi.com

Post reply on HN
Print view View original

31–40 of 115 posts

Re: The gauge broke: devs felt 20% faster with AI, measured 19% slower (2025)

#31

Post by ImprobableTruth »

jiggawatts wrote: post #2

"19 August 2025" This may as well have been written in the stone ages, when we were banging AI rocks together. I just did a ~6 month project in ~2 weeks using a frontier model. I wouldn't even have attempted this kind work a year ago, with or without the AIs available at the time!

>I just did a ~6 month project in ~2 weeks using a frontier model.

Claims like this are hard for me to take seriously because 'good' models have been available since the start of the year. So, if they really 10x one's productivity, then people should be able to have gotten done 5 years worth of work since then, but I've never actually seen anybody show any project like this.

Re: The gauge broke: devs felt 20% faster with AI, measured 19% slower (2025)

#32

Post by [deleted] »

Almondsetat wrote: post #30
wewewedxfgdf wrote: post #14

My two bosses are anti-AI. Whenever I tell them about how awesome AI is, they come back with stories about how they used AI and it couldn't even do anything basic and what it did do had errors. People will always create a world narrative that matches what they already believe. Anti AI people are always quoting these "facts" about how AI reduces productivity even when developers feel it increases productivity - it rei…

>feel Productivity is not a feeling though. Either you show an increased productivity or it doesn't exist

[deleted]

Re: The gauge broke: devs felt 20% faster with AI, measured 19% slower (2025)

#33

Post by wisty »

bitwize wrote: post #5

This study was shown to be flawed at the time; METR has retracted it. And it doesn't take into account current frontier models. AI makes you more productive. This is no longer up for debate. The energy you spend arguing last year's talking points is better spent knuckling down and learning the tools.

Partial agree ...

I suspect:

If you know what you are doing it is a power tool.

If you don't know what you are doing it's also a power tool - if you measure a lot of devs then the bad ones (or anyone having a bad day, or the wrong fit for a project) can make work for everyone else at an outrageous pace.

Re: The gauge broke: devs felt 20% faster with AI, measured 19% slower (2025)

#36

Post by dymk »

Almondsetat wrote: post #30
wewewedxfgdf wrote: post #14

My two bosses are anti-AI. Whenever I tell them about how awesome AI is, they come back with stories about how they used AI and it couldn't even do anything basic and what it did do had errors. People will always create a world narrative that matches what they already believe. Anti AI people are always quoting these "facts" about how AI reduces productivity even when developers feel it increases productivity - it rei…

>feel Productivity is not a feeling though. Either you show an increased productivity or it doesn't exist

Performance review at FAANGs has always been vibes and soft skills.

Re: The gauge broke: devs felt 20% faster with AI, measured 19% slower (2025)

#38

Post by DonHopkins »

Luc Barthelet, who I worked with at EA, is a Mathematica whiz (he later worked at Wolframe Research on Wolfram|Alpha), and he would prototype game ideas in Mathematica, which would render out web pages with animations.

He came up with a fun idea for a racing game renderer: it distorted the perspective transformation a bit, grading depth on a curve, so far away things would linger in the distance a bit longer, then speed up and WHOOSH past you, seeming even faster than they would be photorealisticly!

https://www.mobygames.com/person/29352/luc-barthelet/

https://community.wolfram.com/web/luc

Re: The gauge broke: devs felt 20% faster with AI, measured 19% slower (2025)

#40

Post by mschuetz »

croes wrote: post #21
wewewedxfgdf wrote: post #14

My two bosses are anti-AI. Whenever I tell them about how awesome AI is, they come back with stories about how they used AI and it couldn't even do anything basic and what it did do had errors. People will always create a world narrative that matches what they already believe. Anti AI people are always quoting these "facts" about how AI reduces productivity even when developers feel it increases productivity - it rei…

But this isn’t just a narrative but a study. Limited but a study

Having participated in many studies, I lost much faith in studies. You could study the same thing, and get opposite results depending on how you build the study, which people participate (friends and colleagues will be reluctant to speak against your results), and the bias with which you set everything up. Also, many tasks require learning the tools, and some tools will start to be more productive with expertise than others.
Post reply on HN