Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI slows down open source developers. Peter Naur can teach us why

johnwhiles.com

Post reply on HN
Print view View original

101–110 of 217 posts

Re: AI slows down open source developers. Peter Naur can teach us why

#101

Post by bunderbunder »

narush wrote: post #53
bunderbunder wrote: post #29

> It's a really fabulous study... Ehhhh... not so much. It had serious design flaws in both the protocol and the analysis. This blog post is a fairly approachable explanation of what's wrong with it: https://www.argmin.net/p/are-developers-finally-out-of-a-job

Hey, thanks for linking this! I'm a study author, and I greatly appreciate that this author dug into the appendix and provided feedback so that other folks can read it as well. A few notes if it's helpful: 1. This post is primarily worried about ordering considerations -- I think this is a valid concern. We explicitly call this out in the paper [1] as a factor we can't rule out -- see "Bias from issue completion orde…

Thanks for the response, you make some very points. Sorry, I had missed your response on the original post. I don't know if it was there yet, or because for some reason their blog is configured to only show the first two comments by default. :/ Either way, my bad.

I think my bias as someone who spends too much time looking at social science papers is that the protocol allows for spillover effects that, to me, imply that the results must be interpreted much more cautiously than a lot of people are doing. (And then on top of that I'm trying to be hyper-cautious and skeptical when I see a paper whose conclusions align with my biases on this topic.)

Granted, that sort of thing is my complaint about basically every study on developer productivity when using LLMs that I've seen so far. So I appreciate how difficult this is to study in practice.

Re: AI slows down open source developers. Peter Naur can teach us why

#102

Post by wellpast »

The fact that the devs thought the AI saved them time is no surprise to me… at least at this point in my career.

Developers (people?) in general for some reason just simply cannot see time. It’s why so many people don’t believe in estimation.

What I don’t understand is why. Is this like a general human brain limitation (like not being able to visualize four dimensions, or how some folks don’t have an internal monologue)?

Or is this more psychodynamic or emotional?

It’s been super clear and interesting to me how developers I work with want to believe AI (code generation) is saving them time when it’s clearly obviously not.

Is it just the hope that one day it will? Is it fetishization of AI?

Why in an industry that so requires clarity of thinking and expression (computer processors don’t like ambiguity), can we be so bad at talking about, thinking about… time?

Don’t get me started on the static type enthusiasts who think their strong type system (another seeming fetish) is saving them time.

Re: AI slows down open source developers. Peter Naur can teach us why

#103

Post by sodapopcan »

threetonesun wrote: post #96
sodapopcan wrote: post #90

Earlier quoted context omitted.

This is always my thought whenever I hear the "AI let me build a feature in a codebase I didn't know in a language I didn't know" (which is often, there is at one in these comments). Great, but what have you learned? This is fine for small contributions, I guess, but I don't hear a lot of stories of long-term maintenance. Unpopular opinion, though, I know.

I guess it's a question of how anyone learns. There's some value in typing code, I suppose, but with tab complete that's been gone for a long time. Letting AI write something and then reading it seems as good as copying and pasting from some other source.

I'm not super qualified to answer as I haven't gone deep into AI at all. But from my limited observations I'd say yes and no. You generally aren't copy/pasting entire features, just snippets that you yourself have to string together in a sensible way. Of course there are lots of people who still do this and what's why I find most people in this industry infuriating to work with. It's all good when it's boilerplate, and that's actually my primary use of "AI"—it's essentially been a snippets replacement (and is quite good at that).

Re: AI slows down open source developers. Peter Naur can teach us why

#104

Post by candiddevmike »

yomismoaqui wrote: post #22

Someone on X said that these agentic AI tools (Claude Code, Amp, Gemini Cli) are to programming like the table saw was to hand-made woodworking. It can make some things faster and better than a human with a saw, but you have to learn how to use them right (or you will loose some fingers). I personally find that agentic AI tools make me be more ambitious in my projects, I can tackle some things I didn't tthougth about…

> to programming like the table saw was to hand-made woodworking

This is a ridiculous comparison because the table saw is a precision tool (compared to manual woodworking) when agentic AI is anything but IMO.

Re: AI slows down open source developers. Peter Naur can teach us why

#105

Post by bluefirebrand »

TeMPOraL wrote: post #75
bluefirebrand wrote: post #56

Earlier quoted context omitted.

This runs counter to the starry eyed promises of AI letting people with no experience accomplish things

That promise is true, though, and the two claims are not opposite. The devil is in details, specifically in what you mean by "people" and "accomplish things". If by "people" you mean "general public", and by "accomplish things" you mean solving some immediate problems, that may or may not involve authoring a script or even a small app - then yes, this is already happening, and is a big reason behind the AI hype as it…

> If by "people" you mean "experienced software engineers",

My post is a single sentence and I literally wrote "people with no experience"

Re: AI slows down open source developers. Peter Naur can teach us why

#106

Post by aquariusDue »

colechristensen wrote: post #88
candiddevmike wrote: post #80

Earlier quoted context omitted.

> tell it that it's wrong and it'll go, "You're absolutely right. Let me actually fix it" ...and then it still doesn't actually fix it

Sometimes it does... sometimes. I recently had a nice conversation looking for some reading suggestions from an LLM. The first round of suggestions were superb, some of them I'd already read, some were entirely new and turned out great. Maybe a dozen or so great suggestions. Then it was like squeezing blood from a stone but I did get a few more. After that it was like talking to a babbling idiot. Repeating the same s…

Yeah, when I first heard about "one-shot"ing it felt more like a trick instead of a useful heuristic but with time my experience mimics yours, nowadays I try to one-shot small-ish changes instead of going back and forth.

Re: AI slows down open source developers. Peter Naur can teach us why

#107

Post by mlyle »

candiddevmike wrote: post #80
dawnerd wrote: post #69

Earlier quoted context omitted.

That's what I love about LLMs. You can spot it doesn't know the answer, tell it that it's wrong and it'll go, "You're absolutely right. Let me actually fix it" It scares me how much code is being produced by people without enough experience to spot issues or people that just gave up caring. We're going to be in for wild ride when all the exploits start flowing.

> tell it that it's wrong and it'll go, "You're absolutely right. Let me actually fix it" ...and then it still doesn't actually fix it

So, I recently have done my first couple heavily AI augmented tasks for hobby projects.

I wrote a TON of LVGL code. The result wasn’t perfect for placement, but when I iterated a couple of times, it fixed almost all of the issues. The result is a little hacked together but a bit better than my typical first pass writing UI code. I think this saved me a factor of 10 in time. Next I am going to see how much of the cleanup and factoring of the pile of code it can do.

Next I had it write a bunch of low level code to init hardware. It saved me a little time compared to reading the reference manual, and was more pleasant, but it wasn’t perfectly correct. If I did not have domain expertise I would not have been able to complete the task with the LLM.

Re: AI slows down open source developers. Peter Naur can teach us why

#108

Post by floren »

quantumHazer wrote: post #83
nico wrote: post #8

> They are experienced open source developers, working on their own projects I just started working on a 3-month old codebase written by someone else, in a framework and architecture I had never used before Within a couple hours, with the help of Claude Code, I had already created a really nice system to replicate data from staging to local development. Something I had built before in other projects, and I new that m…

You clearly have not read the study. Problem is developers thought they were 20% faster, but they were actually slower. Anyway from a fast review about your profile you're in conflict of interest about vibe coding, so I will definitely take your opinion with a grain of salt.

> Anyway from a fast review about your profile you're in conflict of interest about vibe coding

Seems to happen every time, doesn't it?

Re: AI slows down open source developers. Peter Naur can teach us why

#109

Post by rsynnott »

bko wrote: post #59
nico wrote: post #8

> They are experienced open source developers, working on their own projects I just started working on a 3-month old codebase written by someone else, in a framework and architecture I had never used before Within a couple hours, with the help of Claude Code, I had already created a really nice system to replicate data from staging to local development. Something I had built before in other projects, and I new that m…

When anecdote and data don't align, it's usually the data that's wrong. Not always the case, but whenever I read about these strained studies or arguments about how AI is actually making people less productive, I can't help but wonder why nearly every programmer I know, myself included, finds value in these tools. I wonder if the same thing happened with higher level programming languages where people argued, you may…

> I can't help but wonder why nearly every programmer I know, myself included, finds value in these tools.

One of the more interesting findings of the study mentioned was that the LLM users, even where use of an LLM had apparently degraded their performance, tended to believe it had enhanced it. Anecdote is a _really_ bad argument against data that shows a _perception_ problem.

> Even if we weren't more "productive", millions prefer to use these tools, so it has to count for something.

I mean, on that basis, so does homeopathy.

Like, it's just one study. It's not the last word. But "my anecdotes disprove it" probably isn't a _terribly_ helpful approach.

Post reply on HN