Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI slows down open source developers. Peter Naur can teach us why

johnwhiles.com

Post reply on HN
Print view View original

81–90 of 217 posts

Re: AI slows down open source developers. Peter Naur can teach us why

#81

Post by pennomi »

antimora wrote: post #50

I'm one of the regular code reviewers for Burn (a deep learning framework in Rust). I recently had to close a PR because the submitter's bug fix was clearly written entirely by an AI agent. The "fix" simply muted an error instead of addressing the root cause. This is exactly what AI tends to do when it can't identify the actual problem. The code was unnecessarily verbose and even included tests for muting the error.…

This is the most frustrating thing LLMs do. They put wide try:catch structures around the code making it impossible to actually track down the source of a problem. I want my code to fail fast and HARD during development so I can solve every problem immediately.

Re: AI slows down open source developers. Peter Naur can teach us why

#82

Post by charcircuit »

omnicognate wrote: post #17

All these studies that show "AI makes developers x% more/less productive" are predicated on the idea that developer "productivity" can be usefully captured in a single objectively measurable number. Just one problem with that...

As long as the true productivity is correlated with that number it should be fine.

Re: AI slows down open source developers. Peter Naur can teach us why

#83

Post by quantumHazer »

nico wrote: post #8

> They are experienced open source developers, working on their own projects I just started working on a 3-month old codebase written by someone else, in a framework and architecture I had never used before Within a couple hours, with the help of Claude Code, I had already created a really nice system to replicate data from staging to local development. Something I had built before in other projects, and I new that m…

You clearly have not read the study. Problem is developers thought they were 20% faster, but they were actually slower. Anyway from a fast review about your profile you're in conflict of interest about vibe coding, so I will definitely take your opinion with a grain of salt.

Re: AI slows down open source developers. Peter Naur can teach us why

#84

Post by colechristensen »

dawnerd wrote: post #69
antimora wrote: post #50

I'm one of the regular code reviewers for Burn (a deep learning framework in Rust). I recently had to close a PR because the submitter's bug fix was clearly written entirely by an AI agent. The "fix" simply muted an error instead of addressing the root cause. This is exactly what AI tends to do when it can't identify the actual problem. The code was unnecessarily verbose and even included tests for muting the error.…

That's what I love about LLMs. You can spot it doesn't know the answer, tell it that it's wrong and it'll go, "You're absolutely right. Let me actually fix it" It scares me how much code is being produced by people without enough experience to spot issues or people that just gave up caring. We're going to be in for wild ride when all the exploits start flowing.

I also get things like this from very experienced engineers working outside their area of expertise. It's obviously less of the completely boneheaded suggestion but still doing exactly the wrong thing suggested by AI that required a person to step in and correct.

Re: AI slows down open source developers. Peter Naur can teach us why

#85

Post by quantumHazer »

Navarr wrote: post #67
Vegenoid wrote: post #45

Earlier quoted context omitted.

I have had similar experiences as you, but this is not the kind of work that the study is talking about: “When open source developers working in codebases that they are deeply familiar with use AI tools to complete a task, they take longer to complete that task” I have anecdotally found this to be true as well, that an LLM greatly accelerates my ramp up time in a new codebase, but then actually leads me astray once I…

> I have anecdotally found this to be true as well, that an LLM greatly accelerates my ramp up time in a new codebase, but then actually leads me astray once I am familiar with the project. If you are unfamiliar with the project, how do you determine that it wasn't leading you astray in the first place? Do you ever revisit what you had done with AI previously to make sure that, once you know your way around, it was d…

no, they just claim that AI coding tools are magic and drink their kool-aid

Re: AI slows down open source developers. Peter Naur can teach us why

#86

Post by narush »

antonvs wrote: post #63
narush wrote: post #27

Hey HN -- study author here! (See previous thread on the paper here [1].) I think this blog post is an interesting take on one specific factor that is likely contributing to slowdown. We discuss this in the paper [2] in the section "Implicit repository context (C.1.5)" -- check it out if you want to see some developer quotes about this factor. > This is why AI coding tools, as they exist today, will generally slow so…

> Early-2025 AI slows down experienced open-source developers. Even that's too general, because it'll depend on what the task is. It's not as if open source developers in general never work on tasks where AI could save time.

We call this over-generalization out specifically in the "We do not provide evidence that:" table in the blog post and paper - I agree there are tasks these developers are likely sped up on with early-2025 tools.

Re: AI slows down open source developers. Peter Naur can teach us why

#87

Post by bko »

adrian_b wrote: post #73
bko wrote: post #59

Earlier quoted context omitted.

When anecdote and data don't align, it's usually the data that's wrong. Not always the case, but whenever I read about these strained studies or arguments about how AI is actually making people less productive, I can't help but wonder why nearly every programmer I know, myself included, finds value in these tools. I wonder if the same thing happened with higher level programming languages where people argued, you may…

TFA says clearly that it is likely that AI will make more productive anyone working on an unfamiliar code base, but make less productive those working on a project they understand well, and it gives reasonable arguments for why this is likely to happen. Moreover, it acknowledges that for programmers working in most companies the first case is much more frequent.

I have written every line of code in the code base I mostly work in and I still find it incredibly valuable. Millions use these tools and a large percentage of them find them useful in their familiar code base.

Again, overwhelming anecdote and millions of users > "study"

Re: AI slows down open source developers. Peter Naur can teach us why

#88

Post by colechristensen »

candiddevmike wrote: post #80
dawnerd wrote: post #69

Earlier quoted context omitted.

That's what I love about LLMs. You can spot it doesn't know the answer, tell it that it's wrong and it'll go, "You're absolutely right. Let me actually fix it" It scares me how much code is being produced by people without enough experience to spot issues or people that just gave up caring. We're going to be in for wild ride when all the exploits start flowing.

> tell it that it's wrong and it'll go, "You're absolutely right. Let me actually fix it" ...and then it still doesn't actually fix it

Sometimes it does... sometimes.

I recently had a nice conversation looking for some reading suggestions from an LLM. The first round of suggestions were superb, some of them I'd already read, some were entirely new and turned out great. Maybe a dozen or so great suggestions. Then it was like squeezing blood from a stone but I did get a few more. After that it was like talking to a babbling idiot. Repeating the same suggestions over and over, failing to listen to instructions, and generally just being useless.

LLMs are great on the first pass but the further you get away from that they degrade into uselessness.

Re: AI slows down open source developers. Peter Naur can teach us why

#89

Post by samtp »

nico wrote: post #8

> They are experienced open source developers, working on their own projects I just started working on a 3-month old codebase written by someone else, in a framework and architecture I had never used before Within a couple hours, with the help of Claude Code, I had already created a really nice system to replicate data from staging to local development. Something I had built before in other projects, and I new that m…

Well that's exactly what it does well at the moment. Boilerplate starter templates, landing pages, throwaway apps, etc. But for projects that need precision like data pipelines, security - it code generated has many subtle flaws that can/will cause giant headaches in your project unless you dig through every line produced

Re: AI slows down open source developers. Peter Naur can teach us why

#90

Post by sodapopcan »

lsy wrote: post #76

Typically debugging, e.g., a tricky race condition in an unfamiliar code base would require adding logging, refactoring library calls, inspecting existing logs, and even rewriting parts of your program to be more modular or understandable. This is part of the theory-building. When you have an AI that says "here is the race condition and here is the code change to make to fix it", that might be "faster" in the immedia…

This is always my thought whenever I hear the "AI let me build a feature in a codebase I didn't know in a language I didn't know" (which is often, there is at one in these comments). Great, but what have you learned? This is fine for small contributions, I guess, but I don't hear a lot of stories of long-term maintenance. Unpopular opinion, though, I know.
Post reply on HN