Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI slows down open source developers. Peter Naur can teach us why

johnwhiles.com

Post reply on HN
Print view View original

151–160 of 217 posts

Re: AI slows down open source developers. Peter Naur can teach us why

#151

Post by tonyedgecombe »

BriggyDwiggs42 wrote: post #117
blake1 wrote: post #94

I think a reasonable summary of the study referenced is that: "AI creates the perception of productivity enhancements far beyond the reality." Even within the study, there were some participants who saw mild improvements to productivity, but most had a significant drop in productivity. This thread is now full of people telling their story about huge productivity gains they made with AI, but none of the comments conte…

I’ve noticed that you can definitely use them to help you learn something, but that your understanding tends to be more abstract and LLM-like that way. You definitely want to mix it up when learning too.

I keep wondering whether the best way to use these tools is to do the work yourself then ask the AI to critique it, to find the bugs, optimisations or missing features.

Re: AI slows down open source developers. Peter Naur can teach us why

#152

Post by dawnerd »

daxfohl wrote: post #123
aquariusDue wrote: post #106

Earlier quoted context omitted.

Yeah, when I first heard about "one-shot"ing it felt more like a trick instead of a useful heuristic but with time my experience mimics yours, nowadays I try to one-shot small-ish changes instead of going back and forth.

I've had some luck in these cases prompting "your context seems to be getting too bloated. summarize this conversation into a prompt that I can feed into a new chat with a fresh context. make sure to include ". Sometimes it works well the first time, and sometimes it spits out a summary where you can see what it is confused about, and you can guide it to create a better summary. Sometimes just having that summary in…

I’ve had really good luck with having gpt generate a todo list that’s very, very detailed. Then having Claude use it to check items off. Still far from perfect but since doing that haven’t run into context issues since I can just start a new chat and feed it the todo (the todo also contains project info).

Re: AI slows down open source developers. Peter Naur can teach us why

#153

Post by pragma_x »

Vegenoid wrote: post #45
nico wrote: post #8

> They are experienced open source developers, working on their own projects I just started working on a 3-month old codebase written by someone else, in a framework and architecture I had never used before Within a couple hours, with the help of Claude Code, I had already created a really nice system to replicate data from staging to local development. Something I had built before in other projects, and I new that m…

I have had similar experiences as you, but this is not the kind of work that the study is talking about: “When open source developers working in codebases that they are deeply familiar with use AI tools to complete a task, they take longer to complete that task” I have anecdotally found this to be true as well, that an LLM greatly accelerates my ramp up time in a new codebase, but then actually leads me astray once I…

Not just new code-bases. I recently used an LLM to accelerate my learning of Rust.

Coming from other programming languages, I had a lot of questions that would be tough to nail down in a Google search, or combing through docs and/or tutorials. In retrospect, it's super fast at finding answers to things that _don't exist_ explicitly, or are implied through the lack of documentation, or exist at the intersection of wildly different resources:

- Can I get compile-time type information of Enum values?

- Can I specialize a generic function/type based on Enum values?

- How can I use macros to reflect on struct fields?

- Can I use an enum without its enclosing namespace, as I can in C++?

- Does rust have a 'with' clause?

- How do I avoid declaring timelines on my types?

- What is an idiomatic way to implement the Strategy pattern?

- What is an idiomatic way to return a closure from a function?

...and so on. This "conversation" happened here and there over the period of two weeks. Not only was ChatGPT up to the task, but it was able to suggest what technologies would get me close to the mark if Rust wasn't built to do what I had in mind. I'm now much more comfortable and competent in the language, but miles ahead of where I would have been without it.

Re: AI slows down open source developers. Peter Naur can teach us why

#154

Post by almatabata »

bko wrote: post #140
almatabata wrote: post #118

Earlier quoted context omitted.

> Interestingly the developers predict that AI will make them faster, and continue to believe that it did make them faster, even after completing the task slower than they otherwise would! In this case clearly anecdotes are not enough. If that quote from the article is accurate, it shows that you cannot trust the developers time perception. I agree, its only one study and we should not take it as the final answer. It…

Or I cannot trust a contrived laboratory setting with it's garden of forking paths. https://mleverything.substack.com/p/garden-of-forking-paths-...

I did not say to trust it. I do not need to trust it.

If I run my own tests on my own codebase I will definitely use some objective time measurement method and a subjective one. I really want to know if there is a big difference.

I really wonder if its just the individuals bias showing. If you are pro-AI you might overestimate one, and if you are against it you might under-estimate it.

Re: AI slows down open source developers. Peter Naur can teach us why

#155

Post by nico »

nico wrote: post #8

> They are experienced open source developers, working on their own projects I just started working on a 3-month old codebase written by someone else, in a framework and architecture I had never used before Within a couple hours, with the help of Claude Code, I had already created a really nice system to replicate data from staging to local development. Something I had built before in other projects, and I new that m…

Some additional notes given the comments in the thread

* I wasn’t trying to be dismissive of the article or the study, just wanted to present a different context in which AI tools do help a lot

* It’s not just code. It also helps with a lot of tasks. For example, Claude Code figured out how to “manually” connect to the AWS cluster that hosted the source db, tested different commands via docker inside the project containers and overall helped immensely with discovery of the overall structure and infrastructure of the project

* My professional experience as a developer, has been that 80-90% of the time, results trump code quality. That’s just the projects and companies I’ve been personally involved with. Mostly saas products in which business goals are usually considered more important than the specifics of the tech stack used. This doesn’t mean that 80-90% of code is garbage, it just means that most of the time readability, maintainability and shipping are more important than DRY, clever solutions or optimizations

* I don’t know how helpful AI is or could be for things that require super clever algorithms or special data structures, or where code quality is incredibly important

* Having said that, the AI tools I’ve used can write pretty good quality code, as long as they are provided with good examples and references, and the developer is on top of properly managing the context

* Additionally, these tools are improving almost on a weekly or monthly basis. My experience with them has drastically changed even in the last 3 months

At the end of the day, AI is not magic, it’s a tool, and I as the developer, am still accountable for the code and results I’m expected to deliver

Re: AI slows down open source developers. Peter Naur can teach us why

#156

Post by tonyedgecombe »

charcircuit wrote: post #72
piker wrote: post #37

My main two attempts at using an “agentic” coding workflow were trying to incorporate an Outlook COM interface into my rust code base and to streamline an existing abstract windows API interaction to avoid copying memory a couple of times. Both wasted tremendous amounts of time and were ultimately abandoned leaving me only slightly more educated about windows development. They make great autocompletion engines but I…

I had the opposite experience. Gemini was able to work with COM and accomplish what I needed despite me never using COM before.

I've done a lot of work with COM over the years and that is the last technology I would trust to an AI. It's very easy to write COM code that appears to work but contains subtle bugs.

Re: AI slows down open source developers. Peter Naur can teach us why

#158

Post by Alex_L_Wood »

munificent wrote: post #113

> The inability of developers to tell if a tool sped them up or slowed them down is fascinating in itself, probably applies to many other forms of human endeavour, and explains things as varied as why so many people think that AI has made them 10 times more productive, why I continue to use Vim, why people drive in London etc. In boating, there's a notion of a "set and drift" which describes how wind and current push…

We all as humans are hardwired to prefer greedy algorithms, basically.

Re: AI slows down open source developers. Peter Naur can teach us why

#159

Post by tonyedgecombe »

afro88 wrote: post #91

I said this when the linked paper was shared and got downvotes: it's based on early 2025 data. My point isn't that it should be completely up to date, but that how we need to consider it in that context. This is pre Claude 4, Claude Code. Pre Gemini 2.5 even. These models are such a big step up from what came previously. Just like we put a (2023) on articles here so they are considered in the right context, so too th…

>This is pre Claude 4, Claude Code. Pre Gemini 2.5 even.

The most interesting point from the article wasn't about how well the AI's worked, rather it was the gap between peoples perception and their actual results.

Re: AI slows down open source developers. Peter Naur can teach us why

#160

Post by jonfw »

bluefirebrand wrote: post #56
Art9681 wrote: post #38

Earlier quoted context omitted.

This is exactly my same take. Any tool an engineer is inexperienced with will slow them down. AI is no different.

This runs counter to the starry eyed promises of AI letting people with no experience accomplish things

AI let's people with no experience accomplish things. People who have experience can create those things without AI. Those experienced folks will likely outperform novices, even when novices leverage AI.

None of these statements are controversial. What we have to establish is- Does the experienced AI builder outperform the experienced manual coder?

Post reply on HN