Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI slows down open source developers. Peter Naur can teach us why

johnwhiles.com

Post reply on HN
Print view View original

161–170 of 217 posts

Re: AI slows down open source developers. Peter Naur can teach us why

#161

Post by ben_w »

bluefirebrand wrote: post #144
helloplanets wrote: post #125

Earlier quoted context omitted.

He addressed your point in the paragraph before that. The paragraph from which you quoted was meant to show the difference between your point and the fact that the original research was indeed measuring software engineers.

My point is that I was very clear about what people I was referring to. No need for all the "if by people you mean" rigamarole

Then your previous point is false, because "X helps Y" doesn't run counter to any promise that "X helps Z".

You said the second. You responded to the first.

Y = [experts]

Z = [noobs]

{Y, Z} ⊆ [all humans]

Re: AI slows down open source developers. Peter Naur can teach us why

#162

Post by jwhiles »

narush wrote: post #27

Hey HN -- study author here! (See previous thread on the paper here [1].) I think this blog post is an interesting take on one specific factor that is likely contributing to slowdown. We discuss this in the paper [2] in the section "Implicit repository context (C.1.5)" -- check it out if you want to see some developer quotes about this factor. > This is why AI coding tools, as they exist today, will generally slow so…

Thanks for the response, and apologies for misrepresenting your results somewhat! I'm probably not going to change the title since I am at heart and polemicist and a sloppy thinker, but I'll update the article to call out this misrepresentation.

That said, I think that what I wrote more or less encompasses three of the factors you call out as being likely to contribute: "High developer familiarity with reposito- ries", "Large and complex repositories", and "Implicit repository context".

I thought more about experimenting on myself, and while I hope to do it - I think it will be very hard to create a controlled enviornment whilst also responding to the demands the job puts on me. I also don't have the luxury of a list of well scoped tasks that could feasibly be completed in a few hours.

Re: AI slows down open source developers. Peter Naur can teach us why

#163

Post by Gormo »

Vegenoid wrote: post #45
nico wrote: post #8

> They are experienced open source developers, working on their own projects I just started working on a 3-month old codebase written by someone else, in a framework and architecture I had never used before Within a couple hours, with the help of Claude Code, I had already created a really nice system to replicate data from staging to local development. Something I had built before in other projects, and I new that m…

I have had similar experiences as you, but this is not the kind of work that the study is talking about: “When open source developers working in codebases that they are deeply familiar with use AI tools to complete a task, they take longer to complete that task” I have anecdotally found this to be true as well, that an LLM greatly accelerates my ramp up time in a new codebase, but then actually leads me astray once I…

> I have anecdotally found this to be true as well, that an LLM greatly accelerates my ramp up time in a new codebase, but then actually leads me astray once I am familiar with the project.

How does using AI impact the amount of time it takes you to become sufficiently familiar with the project to recognize when you are being led astray?

One of the worries I have with the fast ramp-up is that a lot of that ramp-up time isn't just grunt work to be optimized a way, it's active learning, and bypassing too much of it can leave you with an incomplete understanding of the problem domain that slows you down perpetually.

Sometimes, there are real efficiencies to be gained; other times those perceived efficiencies are actually incurring heavy technical debt, and I suspect that overuse of AI is usually the latter.

Re: AI slows down open source developers. Peter Naur can teach us why

#164

Post by bko »

almatabata wrote: post #154
bko wrote: post #140

Earlier quoted context omitted.

Or I cannot trust a contrived laboratory setting with it's garden of forking paths. https://mleverything.substack.com/p/garden-of-forking-paths-...

I did not say to trust it. I do not need to trust it. If I run my own tests on my own codebase I will definitely use some objective time measurement method and a subjective one. I really want to know if there is a big difference. I really wonder if its just the individuals bias showing. If you are pro-AI you might overestimate one, and if you are against it you might under-estimate it.

That's fair, I agree.

Re: AI slows down open source developers. Peter Naur can teach us why

#165

Post by PicassoCTs »

munificent wrote: post #113

> The inability of developers to tell if a tool sped them up or slowed them down is fascinating in itself, probably applies to many other forms of human endeavour, and explains things as varied as why so many people think that AI has made them 10 times more productive, why I continue to use Vim, why people drive in London etc. In boating, there's a notion of a "set and drift" which describes how wind and current push…

I also think that AI written code- is just not read. People hate code-reviews, and actively refuse to read code- because that is hard work, reading into other peoples thoughts and ideas.

This is why pushing for new code, rewrites, new frameworks is so popular. https://www.joelonsoftware.com/2000/04/06/things-you-should-...

So a ton of ai-generated code- is just that, never read. Its generated, tested against test-functions - and thats it. I wouldn't wonder, if some of these devs themselves have only marginal ideas whats in there codebases and why.

Re: AI slows down open source developers. Peter Naur can teach us why

#166

Post by trey-jones »

Doing my own post-mortem of a recent project (the first that I've leaned on "AI" tools to any extent), my feeling was the following:

1. It did not make me faster. I don't know that I expected it to.

2. It's very possible that it made me slower.

3. The quality of my work was better.

Slower and better are related here, because I used these tools more to either check ideas that I had for soundness, or to get some fresh ideas if I didn't have a good one. In many cases the workflow would be: "I don't like that idea, what else do you have for me?"

There were also instances of being led by my tools into a rabbit hole that I eventually just abandoned, so that also contributes to the slowness. This might happen in instances where I'm using "AI" to help cover areas that I'm less of an expert in (and these were great learning experiences). In my areas of expertise, it was much more likely that I would refine my ideas, or the "AI" tool's ideas into something that I was ultimately very pleased with, hence the improved quality.

Now, some people might think that speed is the only metric that matters, and certainly it's harder to quantify quality - but it definitely felt worth it to me.

Re: AI slows down open source developers. Peter Naur can teach us why

#167

Post by [deleted] »

xoralkindi wrote: post #124
nico wrote: post #8

> They are experienced open source developers, working on their own projects I just started working on a 3-month old codebase written by someone else, in a framework and architecture I had never used before Within a couple hours, with the help of Claude Code, I had already created a really nice system to replicate data from staging to local development. Something I had built before in other projects, and I new that m…

How are you confident in the code, coding style and practices simply because the LLM says so. How do you know it is not hallucinating since you don't understand the codebase?

[deleted]

Re: AI slows down open source developers. Peter Naur can teach us why

#168

Post by jpc0 »

trey-jones wrote: post #166

Doing my own post-mortem of a recent project (the first that I've leaned on "AI" tools to any extent), my feeling was the following: 1. It did not make me faster. I don't know that I expected it to. 2. It's very possible that it made me slower. 3. The quality of my work was better. Slower and better are related here, because I used these tools more to either check ideas that I had for soundness, or to get some fresh…

I do this a lot and absolutely think it might even improve it, and this is why I like the current crop of AIs that are more likely to be argumentative and not just capitulate.

I will ask the AI for an idea and then start blowing holes in its idea, or will ask it to do the same for my idea.

And I might end up not going with it’s idea regardless but it got me thinking about things I wouldn’t have thought about.

Effectively its like chatting to a coworker that has a reasonable idea about the domain and can bounce ideas around.

Re: AI slows down open source developers. Peter Naur can teach us why

#169

Post by tjr »

PicassoCTs wrote: post #165
munificent wrote: post #113

> The inability of developers to tell if a tool sped them up or slowed them down is fascinating in itself, probably applies to many other forms of human endeavour, and explains things as varied as why so many people think that AI has made them 10 times more productive, why I continue to use Vim, why people drive in London etc. In boating, there's a notion of a "set and drift" which describes how wind and current push…

I also think that AI written code- is just not read. People hate code-reviews, and actively refuse to read code- because that is hard work, reading into other peoples thoughts and ideas. This is why pushing for new code, rewrites, new frameworks is so popular. https://www.joelonsoftware.com/2000/04/06/things-you-should-... So a ton of ai-generated code- is just that, never read. Its generated, tested against test-fun…

I have mostly worked in aerospace software, and find this rather horrifying. I suppose, if your tests are in fact good and comprehensive enough, there could be a logical argument for not needing to understand the code, but if we're talking people's safety in the hands of your software, I don't know if there is any number of tests I would accept in exchange for willingly giving up understanding of the code.

Re: AI slows down open source developers. Peter Naur can teach us why

#170

Post by delusional »

la_fayette wrote: post #132
mlyle wrote: post #107

Earlier quoted context omitted.

So, I recently have done my first couple heavily AI augmented tasks for hobby projects. I wrote a TON of LVGL code. The result wasn’t perfect for placement, but when I iterated a couple of times, it fixed almost all of the issues. The result is a little hacked together but a bit better than my typical first pass writing UI code. I think this saved me a factor of 10 in time. Next I am going to see how much of the clea…

When you argued that it saved you time by a factor of 10, have you even measured that properly? I initially also had the feeling that LLMs save me time, but in the end it didn't. I roughly compared my performance to past performance by the amount of stories done and LLMs made me slower even if I thought I am saving time... From several month of deep work with LLMs I think they are amazing pattern matchers, but not pr…

There was a recent study that found that LLM users in general tend to feel like they were more productive with AI while actually being less productive.
Post reply on HN