> I don’t see how you can argue it’s not.
Yes you do, you already made the argument when you pointed out the “team” size and makeup was completely unspecified, therefore the number is not an estimate, it’s just a number.
When you call it an “estimate” you are adding additional unsupported specificity to something that was explicitly stated as being hand-wavy to make an obviously rhetorical point. Are you saying you can’t understand the point being made?
My 10x is based on my experience doing projects with Claude. I also said “some” tasks, not all tasks, and I didn’t specify which tasks, and I clarified that my number is made up, which is why my number is also not an estimate of anything. There are some tasks that Claude can do 10x faster than me, and there are some tasks that it can do 100x faster than me, and there are some tasks I can do faster than Claude... for now... More importantly for me personally, Claude makes starting projects and using tech I don’t already know easier; it’s lower effort, regardless of speed.
The paper is interesting and a valid data point, but I don’t think it proves your point. I’ll respond with a few thoughts.
First, the dev’s self estimate of AI productivity speedup was +20%, even though their measured productivity was -20%. This may relate to the effort and not the speed, and it’s important to note that this is a gray zone the paper didn’t explore, and something that can be true on both sides. I can be “faster” at developing and still take the same or longer wall clock time. Measuring the time doesn’t capture how the time was spent, nor the qualities of that time.
Second, this study was done a year ago. That’s an eternity in AI land, and everyone noticed Claude and other models getting substantially better at code writing last fall, plus workflows and tooling are improving even faster than that. There’s every reason to believe the outcome of the exact same study might be different this year than it was last year.
Third, this study is explicitly biased toward large projects, and large projects are, even today, more difficult to find the productivity boosts with. I find Claude absolutely amazing at starting new projects, and absolutely terrible at working in large code bases that don’t fit in context. When I say Claude makes me 10x faster at some projects, I’m referring to something like setting up a new CRUD app when I don’t know much about setting up a database and web server backend, or writing a graphics app in Vulkan when I’ve only used OpenGL. Doing stuff like that, having Claude help me with tech stacks I don’t know, absolutely is many multiples faster than doing it on my own, and the paper link you’ve shared doesn’t address that use of AI at all.
Note specifically the paper says they are not demonstrating or claiming that “AI systems do not currently speed up many or most software developers”, and they have not demonstrated or claimed that “AI systems in the near future will not speed up developers in our exact setting”. It might be a mistake on your part to try to use this as some kind of evidence that AI isn’t speeding devs up.