Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI slows down open source developers. Peter Naur can teach us why

johnwhiles.com

Post reply on HN
Print view View original

121–130 of 217 posts

Re: AI slows down open source developers. Peter Naur can teach us why

#121

Post by davidclark »

nico wrote: post #8

> They are experienced open source developers, working on their own projects I just started working on a 3-month old codebase written by someone else, in a framework and architecture I had never used before Within a couple hours, with the help of Claude Code, I had already created a really nice system to replicate data from staging to local development. Something I had built before in other projects, and I new that m…

> That PR, would have taken me at least a couple of days and up to 2 weeks to fully manually write out and test

What is your accuracy on software development estimates? I always see these productivity claims matched again “It would’ve taken me” timelines.

But, it’s never examined if we’re good at estimating. I know I am not good at estimates.

It’s also never examined if the quality of the PR is the same as it would’ve been. Are you skipping steps and system understanding which let you go faster, but with a higher % chance of bugs? You can do that without AI and get the same speed up.

Re: AI slows down open source developers. Peter Naur can teach us why

#122

Post by thinkingemote »

munificent wrote: post #113

> The inability of developers to tell if a tool sped them up or slowed them down is fascinating in itself, probably applies to many other forms of human endeavour, and explains things as varied as why so many people think that AI has made them 10 times more productive, why I continue to use Vim, why people drive in London etc. In boating, there's a notion of a "set and drift" which describes how wind and current push…

Exactly! Waze the navigation app tends to route users on longer routes but which feels more fast. When driving we perceive our journey as fast or slow not by the actual length but by our memories of what happened. Waze knows human drivers are happier with driving a route that may be longer in time and distance of they feel like they are making progress with the twists and turns.

Ai tools makes programming feel easier. That it might be actually less productive is interesting but we humans prefer the easier shortcuts. Our memories of coding with AI tells us that we didn't struggle and therefore we made progress.

Re: AI slows down open source developers. Peter Naur can teach us why

#123

Post by daxfohl »

aquariusDue wrote: post #106
colechristensen wrote: post #88

Earlier quoted context omitted.

Sometimes it does... sometimes. I recently had a nice conversation looking for some reading suggestions from an LLM. The first round of suggestions were superb, some of them I'd already read, some were entirely new and turned out great. Maybe a dozen or so great suggestions. Then it was like squeezing blood from a stone but I did get a few more. After that it was like talking to a babbling idiot. Repeating the same s…

Yeah, when I first heard about "one-shot"ing it felt more like a trick instead of a useful heuristic but with time my experience mimics yours, nowadays I try to one-shot small-ish changes instead of going back and forth.

I've had some luck in these cases prompting "your context seems to be getting too bloated. summarize this conversation into a prompt that I can feed into a new chat with a fresh context. make sure to include ".

Sometimes it works well the first time, and sometimes it spits out a summary where you can see what it is confused about, and you can guide it to create a better summary. Sometimes just having that summary in its context gets it over the hump and you can just say "actually I'm going to continue with you; please reference this summary going forward", and sometimes you actually do have to restart the LLM with the new context. And of course sometimes there's nothing that works at all.

Re: AI slows down open source developers. Peter Naur can teach us why

#124

Post by xoralkindi »

nico wrote: post #8

> They are experienced open source developers, working on their own projects I just started working on a 3-month old codebase written by someone else, in a framework and architecture I had never used before Within a couple hours, with the help of Claude Code, I had already created a really nice system to replicate data from staging to local development. Something I had built before in other projects, and I new that m…

How are you confident in the code, coding style and practices simply because the LLM says so. How do you know it is not hallucinating since you don't understand the codebase?

Re: AI slows down open source developers. Peter Naur can teach us why

#125

Post by helloplanets »

bluefirebrand wrote: post #105
TeMPOraL wrote: post #75

Earlier quoted context omitted.

That promise is true, though, and the two claims are not opposite. The devil is in details, specifically in what you mean by "people" and "accomplish things". If by "people" you mean "general public", and by "accomplish things" you mean solving some immediate problems, that may or may not involve authoring a script or even a small app - then yes, this is already happening, and is a big reason behind the AI hype as it…

> If by "people" you mean "experienced software engineers", My post is a single sentence and I literally wrote "people with no experience"

He addressed your point in the paragraph before that. The paragraph from which you quoted was meant to show the difference between your point and the fact that the original research was indeed measuring software engineers.

Re: AI slows down open source developers. Peter Naur can teach us why

#126

Post by imiric »

alganet wrote: post #98

This idea that some developers have some "mental model" and others not is an extraordinary claim, and I don't see extraordinary evidence. It sounds like a good thing, right? "Wow, mental model. I want that, I want to be good and have big brain", which encourages you to believe the bullshit. The truth is, this paper is irrelevant and a waste of time. It only serves the purpose of creating discussion around the subject…

You couldn't be more wrong. If you've ever programmed, or worked with programmers, that is not an extraordinary claim at all, but a widely accepted fact.

A mental model of the software is what allows a programmer to intuitively know why the software is behaving a certain way, or what the most optimal design for a feature would be. In the vast majority of cases these intuitions are correct, and other programmers should pay attention to them. This ability is what separates those with a mental model and those without.

On the other hand, LLMs are unable to do this, and are usually not used in ways that help build a mental model. At best, they can summarize the design of a system or answer questions about its behavior, which can be helpful, but a mental model is an abstract model of the software, not a textual summary of its design or behavior. Those neural pathways can only be activated by natural learning and manual programming.

Re: AI slows down open source developers. Peter Naur can teach us why

#127

Post by Macha »

antimora wrote: post #50

I'm one of the regular code reviewers for Burn (a deep learning framework in Rust). I recently had to close a PR because the submitter's bug fix was clearly written entirely by an AI agent. The "fix" simply muted an error instead of addressing the root cause. This is exactly what AI tends to do when it can't identify the actual problem. The code was unnecessarily verbose and even included tests for muting the error.…

I recently reviewed a MR from a coworker. There was a test that was clearly written by AI, except I guess however he prompted it, it gave some rather poor variable names like "thing1", "thing2", etc. in test cases. Basically, these were multiple permutations of data that all needed to be represented in the result set. So I asked for them to be named distinctively, maybe by what makes them special.

It's clear he just took that feedback and asked the AI to make the change, and it came up with a change that gave them all very long, very unique names, that just listed all the unique properties in the test case. But to the extent that they sort of became noise.

It's clear writing the PR was very fast for that developer, I'm sure they felt they were X times faster than writing it themselves. But this isn't a good outcome for the tool either. And I'm sure if they'd reviewed it to the extent I did, a lot of that gained time would have dissipated.

Re: AI slows down open source developers. Peter Naur can teach us why

#128

Post by rustyminnow »

doug_durham wrote: post #112
antimora wrote: post #50

I'm one of the regular code reviewers for Burn (a deep learning framework in Rust). I recently had to close a PR because the submitter's bug fix was clearly written entirely by an AI agent. The "fix" simply muted an error instead of addressing the root cause. This is exactly what AI tends to do when it can't identify the actual problem. The code was unnecessarily verbose and even included tests for muting the error.…

You should be rejecting the PR because the fix was insufficient, not because it was AI agent written. Bad code is bad code regardless of the source. I think the fixation on how the code was generated is not productive.

> You should be rejecting the PR because the fix was insufficient

I mean they probly could've articulated it your way, but I think that's basically what they did... they point out the insufficient "fix" later, but the root cause of the "fix" was blind trust in AI output, so that's the part of the story they lead with.

Re: AI slows down open source developers. Peter Naur can teach us why

#129

Post by conartist6 »

uludag wrote: post #11

Great article and I was having very similar thoughts with regards to this productivity study and the "Programming as Theory Building" paper. I'm starting to be convinced that if you are the original author of a program and still have the program's context in the head, you are the asymptote to which any and all AI systems will approach but never surpass: maybe not in terms of raw coding speed, but in terms of understa…

Thanks, <3

Re: AI slows down open source developers. Peter Naur can teach us why

#130

Post by i_love_retros »

The AI hype will die off just like block chain and web3. LLMs are a solution in search of a problem.

All the VCs are gonna lose a ton of money! OpenAI will be NopenAI, relegated to the dustbin of history.

We never asked for this, nobody wants it.

Companies using AI and promoting it in their products will be seen as tacky and cheap. Just like developers and artists that use it.

Post reply on HN