AI slows down open source developers. Peter Naur can teach us why • Hacker News Board

Re: AI slows down open source developers. Peter Naur can teach us why

#211

Post by jplusequalt » Tue, Jul 15, 2025, 1:50 PM UTC

rosspackard wrote: post #44

jplusequalt wrote: post #30

Earlier quoted context omitted.

>One mediocre paper/study (it should not even be called that with all the bias and sample size issues) Can you bring up any specific issues with the metr study? Alternatively, can you site a journal that critiques it?

It was just published. Too new for someone to conduct a direct study to critique and journals don't just publish critiques anyway. It would have to be a study that disputes the results. They used 16 developers. The confidence intervals are wide and a few atypical issues per dev could swing the headline figure. Veteran maintainers on projects they know inside-out. This is a bias. Devs supplied the issue list (then ran…

TBH, most of your points are a bit of a reach.

>They used 16 developers. The confidence intervals are wide and a few atypical issues per dev could swing the headline figure

This is reasonable, but there have been enough anecdotal evidence from developers over the last 3 years for me to believe the data is measuring something real.

>Veteran maintainers on projects they know inside-out. This is a bias

I think this is complete BS. The study was trying to measure the real world impact of these tools with experienced developers. I think having them try them out on greenfield work, or a code-base they are not familiar with, makes it harder to measure this.

Also, let's be honest--if the study showed that LLMs DID increase productivity on greenfield work, does that even matter? How many developers out there are starting greenfield projects on a weekly basis? I'd argue very few. So if the study is suggesting that experienced developers are better working on code they're already familiar with without the assistance of an LLM, then that means the vast majority of software development work could be better off without LLMs.

>Devs supplied the issue list (then randomized) which still leads to subtle self-selection bias. Maintainers may pick tasks they enjoy or that showcase deep repo knowledge—exactly where AI probably has least marginal value

Again, for MANY developers, they are going to have deep repo knowledge. If they're not faster with LLMs, despite the knowledge, why use them? You're trying to prop up this as bias against the study, but IMO you're missing the point.

Re: AI slows down open source developers. Peter Naur can teach us why

#212

Post by antonvs » Tue, Jul 15, 2025, 1:51 PM UTC

raincole wrote: post #65

antonvs wrote: post #54

Earlier quoted context omitted.

> You've missed the point of the article Sadly clickbait headlines like the OP, "AI slows down open source developers," spread this misinformation, ensuring that a majority of people will have the same misapprehension.

Which is a good thing for people who are currently benefiting from AI, though. The slower other programmers adopt AI, the more edge those who are proficient with it have. It took me an embarrassingly long time to realize a simple fact: using AI well is a shallow skill that everyone can learn in days or even hours if they want. And then my small advantage of knowing AI tools will disappear. Since the realization I've…

I'd recommend spending more effort on improving your actual skills. The advantage of being good at what you do tends to be quite stable and more beneficial.

Re: AI slows down open source developers. Peter Naur can teach us why

#213

Post by sltr » Tue, Jul 15, 2025, 2:37 PM UTC

A couple of months ago I put forth Naur's program theory as an argument why LLM's can't replace human developers:

> LLMs as they currently exist cannot master a theory, design, or mental construct because they don't remember beyond their context window. Only humans can can gain and retain program theory.

https://news.ycombinator.com/item?id=44114631

Re: AI slows down open source developers. Peter Naur can teach us why

#214

Post by PleasureBot » Tue, Jul 15, 2025, 7:27 PM UTC

sodapopcan wrote: post #90

lsy wrote: post #76

Typically debugging, e.g., a tricky race condition in an unfamiliar code base would require adding logging, refactoring library calls, inspecting existing logs, and even rewriting parts of your program to be more modular or understandable. This is part of the theory-building. When you have an AI that says "here is the race condition and here is the code change to make to fix it", that might be "faster" in the immedia…

This is always my thought whenever I hear the "AI let me build a feature in a codebase I didn't know in a language I didn't know" (which is often, there is at one in these comments). Great, but what have you learned? This is fine for small contributions, I guess, but I don't hear a lot of stories of long-term maintenance. Unpopular opinion, though, I know.

For the same reasons as you said I don't understand its use in our side projects. Maybe I'm alone in this, but I feel like the entire point of a side project is to learn some new tech, framework, etc. If you just let an LLM do the work for you, you don't actually learn anything about the underlying tech, so what was the point of the whole thing? I think LLMs abuse our dopamine system that usually activates when we get our side projects working and allows us to feel that with very little effort. Of course at the cost of not learning anything other than how to prompt an LLM.

Re: AI slows down open source developers. Peter Naur can teach us why

#215

Post by cadamsdotcom » Wed, Jul 16, 2025, 5:56 AM UTC

The study doesn’t provide data that can be extrapolated in any way to any statement about anybody slowing down or speeding up.

1. They used Cursor, which makes you spend your whole day saying “yes” to code changes. Cursor is the windows vista “cancel or allow” of agentic coding.

2. Every user was a Cursor beginner.

The study measured developers’ effectiveness at writing code during the ramp up period while they were learning to use a bad tool.

Re: AI slows down open source developers. Peter Naur can teach us why

#216

Post by BriggyDwiggs42 » Thu, Jul 17, 2025, 10:52 PM UTC

tonyedgecombe wrote: post #151

BriggyDwiggs42 wrote: post #117

Earlier quoted context omitted.

I’ve noticed that you can definitely use them to help you learn something, but that your understanding tends to be more abstract and LLM-like that way. You definitely want to mix it up when learning too.

I keep wondering whether the best way to use these tools is to do the work yourself then ask the AI to critique it, to find the bugs, optimisations or missing features.

I think in fuzzier fields that’s a very bad idea, but in more hard sciency fields works alright. I’ve tried having it test my understanding on philosophy and it’s incredibly sycophantic.

Re: AI slows down open source developers. Peter Naur can teach us why

#217

Post by BriggyDwiggs42 » Thu, Jul 17, 2025, 10:54 PM UTC

daxfohl wrote: post #149

BriggyDwiggs42 wrote: post #117

Earlier quoted context omitted.

I’ve noticed that you can definitely use them to help you learn something, but that your understanding tends to be more abstract and LLM-like that way. You definitely want to mix it up when learning too.

I've also had bad results with hallucinations there. I was trying to learn more about multi-dimensional qubit algorithms, and spent a whole day learning a bunch of stuff that was fascinating but plain wrong. I only figured out it was wrong at the end of the day when I tried to do a simulation and the results weren't consistent. Early in the chat it substituted a `-1` for an `i`, and everything that followed was garba…

Yup, I think a mixed approach is probably best. It’s really good for targeted, specific questions but best used with reference to docs or something that you can check its work with anytime you’re even slightly suspicious.