Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI slows down open source developers. Peter Naur can teach us why

johnwhiles.com

Post reply on HN
Print view View original

71–80 of 217 posts

Re: AI slows down open source developers. Peter Naur can teach us why

#71

Post by throwaway284927 »

sumeno wrote: post #58
throwaway284927 wrote: post #51

Earlier quoted context omitted.

That is not true, usage of AI was decided randomly. From the paper: "To directly measure the impact of AI tools on developer productivity, we conduct a randomized controlled trial by having 16 developers complete 246 tasks (2.0 hours on average) on well-known open-source repositories (23,000 stars on average) they regularly contribute to. Each task is randomly assigned to allow or disallow AI usage, and we measure ho…

Directly from the paper: > If AI is allowed, developers can use any AI tools or models they choose, including no AI tooling if they expect it to not be helpful. If AI is not allowed, no generative AI tooling can be used. AI is allowed not required

True, my bad, I didn't read you correctly. What you said was true.

I do believe however that it's important to emphasize the fact that they didn't got to choose in general, though, which I think your wording (even though it is correct) does not make evident.

Re: AI slows down open source developers. Peter Naur can teach us why

#72

Post by charcircuit »

piker wrote: post #37

My main two attempts at using an “agentic” coding workflow were trying to incorporate an Outlook COM interface into my rust code base and to streamline an existing abstract windows API interaction to avoid copying memory a couple of times. Both wasted tremendous amounts of time and were ultimately abandoned leaving me only slightly more educated about windows development. They make great autocompletion engines but I…

I had the opposite experience. Gemini was able to work with COM and accomplish what I needed despite me never using COM before.

Re: AI slows down open source developers. Peter Naur can teach us why

#73

Post by adrian_b »

bko wrote: post #59
nico wrote: post #8

> They are experienced open source developers, working on their own projects I just started working on a 3-month old codebase written by someone else, in a framework and architecture I had never used before Within a couple hours, with the help of Claude Code, I had already created a really nice system to replicate data from staging to local development. Something I had built before in other projects, and I new that m…

When anecdote and data don't align, it's usually the data that's wrong. Not always the case, but whenever I read about these strained studies or arguments about how AI is actually making people less productive, I can't help but wonder why nearly every programmer I know, myself included, finds value in these tools. I wonder if the same thing happened with higher level programming languages where people argued, you may…

TFA says clearly that it is likely that AI will make more productive anyone working on an unfamiliar code base, but make less productive those working on a project they understand well, and it gives reasonable arguments for why this is likely to happen.

Moreover, it acknowledges that for programmers working in most companies the first case is much more frequent.

Re: AI slows down open source developers. Peter Naur can teach us why

#74

Post by marcosdumay »

lupire wrote: post #25
sureglymop wrote: post #9

Earlier quoted context omitted.

That doesn't surprise me at all. Isn't software engineering in essence about being constantly confronted with new problems to solve and having to come up with a sufficient one on the fly? It seems very hard to estimate this, even if you know yourself well.

They were 20% underestimating how long it took them to do a 1-8 hr task that they had just completed. It's like Tog's study that people think Keyboard is faster than the mouse even when they are faster with the mouse. Because they are measuring how they feel, not what is actually happening. https://www.asktog.com/TOI/toi06KeyboardVMouse1.html

That is a very weird set of findings.

This one in particular:

> It takes two seconds to decide upon which special-function key to press.

seems to indicate the study was done on people with no familiarity at all with the software they were testing.

Either way, I don't think there is any evidence out there supporting that either of keyboard-only or mouse-only is faster or equivalent to keyboard+mouse for well known GUIs.

Re: AI slows down open source developers. Peter Naur can teach us why

#75

Post by TeMPOraL »

bluefirebrand wrote: post #56
Art9681 wrote: post #38

Earlier quoted context omitted.

This is exactly my same take. Any tool an engineer is inexperienced with will slow them down. AI is no different.

This runs counter to the starry eyed promises of AI letting people with no experience accomplish things

That promise is true, though, and the two claims are not opposite. The devil is in details, specifically in what you mean by "people" and "accomplish things".

If by "people" you mean "general public", and by "accomplish things" you mean solving some immediate problems, that may or may not involve authoring a script or even a small app - then yes, this is already happening, and is a big reason behind the AI hype as it is.

If by "people" you mean "experienced software engineers", and by "accomplish things" you mean meaningful contributions to a large software product, measured by high internal code and process quality standards, then no - AI tools may not help with that directly, though chances are greater when you have enough experience with those tools to reliably give them right context and steer away from failure modes.

Still, solving one-off problems != incremental improvements to a large system.

Re: AI slows down open source developers. Peter Naur can teach us why

#76

Post by lsy »

Typically debugging, e.g., a tricky race condition in an unfamiliar code base would require adding logging, refactoring library calls, inspecting existing logs, and even rewriting parts of your program to be more modular or understandable. This is part of the theory-building.

When you have an AI that says "here is the race condition and here is the code change to make to fix it", that might be "faster" in the immediate sense, but it means you aren't understanding the program better or making it easier for anyone else to understand. There is also the question of whether this process is sustainable: does an AI-edited program eventually fall so far outside what is "normal" for a program that the AI becomes unable to model correct responses?

Re: AI slows down open source developers. Peter Naur can teach us why

#77

Post by quantumHazer »

gjsman-1000 wrote: post #4

What I thought was fascinating, and should be a warning sign to everyone here: Before beginning the study, the average developer expected about a 20% productivity boost. After ending the study, the average developer (potentially: you) believed they actually were 20% more productive. In reality, they were 0% more productive at best, and 40% less productive at worst. Think about what it would be like to be that develop…

This should really be top comment. The problem is this tools can really give us some value in certain type of areas, but they are not like they are marketed.

Re: AI slows down open source developers. Peter Naur can teach us why

#78

Post by yomismoaqui »

bgwalter wrote: post #26
yomismoaqui wrote: post #22

Someone on X said that these agentic AI tools (Claude Code, Amp, Gemini Cli) are to programming like the table saw was to hand-made woodworking. It can make some things faster and better than a human with a saw, but you have to learn how to use them right (or you will loose some fingers). I personally find that agentic AI tools make me be more ambitious in my projects, I can tackle some things I didn't tthougth about…

"You are using it wrong!" This is insulting to all pre-2023 open source developers, who produced the entire stack that the "AI" robber barons use in their companies. It is even more insulting because no actual software of value has been demonstrably produced using "AI".

> It is even more insulting because no actual software of value has been demonstrably produced using "AI".

Claude Code and Amp (equivalent from Sourcegraph) are created by humans using these same tools to add new features and fix bugs.

Having used both tools for some weeks I can tell you that they provide a great value to me, enough that I see paying $100 monthly as a bargain related to that value.

Edit: typo

Re: AI slows down open source developers. Peter Naur can teach us why

#79

Post by andix »

What I noticed: AI development constantly breaks my flow. It makes me more tired, and I work for shorter time periods on coding.

It's a myth that you can code a whole day long. I usually do intervals of 1-3 hours for coding, with some breaks in between. Procrastination can even happen on work related things, like reading other project members code/changes for an hour. It has a benefit to some extent, but during this time I don't get my work done.

Agentic AI works the best for me. Small refactoring tasks on a selected code snippet can be helpful, but isn't a huge time saver. The worst are AI code completions (first version Copilot style), they are much more noise then help.

Re: AI slows down open source developers. Peter Naur can teach us why

#80

Post by candiddevmike »

dawnerd wrote: post #69
antimora wrote: post #50

I'm one of the regular code reviewers for Burn (a deep learning framework in Rust). I recently had to close a PR because the submitter's bug fix was clearly written entirely by an AI agent. The "fix" simply muted an error instead of addressing the root cause. This is exactly what AI tends to do when it can't identify the actual problem. The code was unnecessarily verbose and even included tests for muting the error.…

That's what I love about LLMs. You can spot it doesn't know the answer, tell it that it's wrong and it'll go, "You're absolutely right. Let me actually fix it" It scares me how much code is being produced by people without enough experience to spot issues or people that just gave up caring. We're going to be in for wild ride when all the exploits start flowing.

> tell it that it's wrong and it'll go, "You're absolutely right. Let me actually fix it"

...and then it still doesn't actually fix it

Post reply on HN