Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Did Claude increase bugs in rsync?

alexispurslane.github.io

Post reply on HN
Print view View original

111–120 of 611 posts

Re: Did Claude increase bugs in rsync?

#111

Post by OptionOfT »

aesthesia wrote: post #95

I don't have a dog in this fight, but a few points that look a little suspicious: - The release with the highest number of attributed bugs is the release _right before_ the first release with Claude-coauthored commits, released in January; is there a chance that unattributed LLM-authored commits made it into this release? - The release attribution methodology is not great, since it will tend to attribute bugs introdu…

You can use LLMs in multiple ways, from very hands on to make local changes to completely hands-off.

I've seen plenty of code that was LLM generated but the commit message itself did not have the co-author attached to it. This only seems to happen when someone's interface to the codebase is completely though Claude/Codex/..., and those are usually the most verbose commits, and yet they say the least, because they just summarize the code changes, not the why.

On the other hand I've seen developers using Claude as a tool. They have VSCode open and a terminal window with Claude and go back and forth, ensuring they write correct code, and leave the plumbing to Claude.

So maybe the author of the code started off small and it grew over time?

Re: Did Claude increase bugs in rsync?

#113

Post by KaiShips »

faitswulff wrote: post #22

> The analysis uses a single metric: bugs per 10 commits (bugs/10c). Bugs per commit as a metric papers over severity, both in terms of security severity as well as the effect on the user. A mislabeled button has the same weight as the entire app crashing in this framework.

[flagged]

Re: Did Claude increase bugs in rsync?

#115

Post by amiga386 »

julianeon wrote: post #109
matheusmoreira wrote: post #104

Earlier quoted context omitted.

> You’re just going to make people disable Claude attribution on their commits to avoid drama. People should be doing this regardless of drama. No reason to provide free advertising for trillion dollar corporations. Generated-by trailers are only relevant when contributing to third party projects, in that case disclosure is polite.

If Claude is actually good enough to commit to rsync, of course I'm going to look at that and think "it's good enough for my side project too." And (benefit to companies aside) that is info it is useful to know, if it's true.

Yeah, this is why it's obnoxious and this is why scummy marketers do it. If you don't aggressively turn it off, they leech an implicit endorsement out of you.

- Sent from my iPhone

Re: Did Claude increase bugs in rsync?

#116

Post by logicprog »

hypfer wrote: post #76
dang wrote: post #68

Earlier quoted context omitted.

This submission was heavily flagged, presumably because the article sounded like genai. But the article now says the following: > After posting this on Hacker News and recieving almost no substantive input, discussion, or response on the actual content of the article, I decided to rewrite all of the prose in my own voice. I've therefore turned off the flags and hopefully people can actually now discuss the claims/fin…

> I decided to rewrite all of the prose in my own voice. Soo... it didn't just sound like genai but was genai? ___ Huh. From the article: > If anyone complains about my verbosity or sentence structure — as they usually do, which is the reason I originally let the AI write the prose, among other reasons obsoleted by templating — they can go fuck themselves. This is kinda sad, honestly. But also should show the author…

> .. what are those em-dashes doing there though?

You're literally doing exactly the bullying I was trying to avoid, even while denouncing it. I like em-dashes. I have AuDHD, and they help me represent how I think.

Re: Did Claude increase bugs in rsync?

#117

Post by jonquark »

logicprog wrote: post #103
aesthesia wrote: post #95

I don't have a dog in this fight, but a few points that look a little suspicious: - The release with the highest number of attributed bugs is the release _right before_ the first release with Claude-coauthored commits, released in January; is there a chance that unattributed LLM-authored commits made it into this release? - The release attribution methodology is not great, since it will tend to attribute bugs introdu…

Your first and second points seem to contradict each other because if all of the bugs for 3.4.1 should be attributed to 3.4.0, that pushes the timetable back even further that unattributed LLM commits would have to have been being committed to the project, which just makes your point even more absurd. Which brings me to my overall response, which is that there is absolutely no evidence, and nothing even intimating th…

Isn't the metric that you've used "bugs per commit ~ per new line of code" going to miss the issue?

All code is technical debt.

If rsync releases used to have 500 lines changed and 5 bugs in and AI-powered rsync releases have 50000 lines and 500 bugs, it's the same bugs/line but much worse experience for the user?

I've not looked into the details of this case and I do use AI assistance coding at work but in my experience, the problem is that it's too easy to write lots of code and therefore hard to review the huge volumes of code and this analysis will ignore that?

edit: actually your table shows there weren't unusually large numbers of commits in this release, so perhaps my initial skepticism shows a bias I have?

Re: Did Claude increase bugs in rsync?

#118

Post by aesthesia »

logicprog wrote: post #103
aesthesia wrote: post #95

I don't have a dog in this fight, but a few points that look a little suspicious: - The release with the highest number of attributed bugs is the release _right before_ the first release with Claude-coauthored commits, released in January; is there a chance that unattributed LLM-authored commits made it into this release? - The release attribution methodology is not great, since it will tend to attribute bugs introdu…

Your first and second points seem to contradict each other because if all of the bugs for 3.4.1 should be attributed to 3.4.0, that pushes the timetable back even further that unattributed LLM commits would have to have been being committed to the project, which just makes your point even more absurd. Which brings me to my overall response, which is that there is absolutely no evidence, and nothing even intimating th…

Sorry, I should have said this explicitly in the original comment: I think you're likely _correct_ that there isn't a clear increase in the rate of bugs attributable to LLM-authored code in rsync. Your analysis provides evidence in this direction; these are just the things that made me go "hmm". They're not accusations or claims that the conclusion is invalid. But they're definitely things to be curious about.

Regarding unlabeled LLM-authored commits, I don't think it's unreasonable in general to think that an open-source project might have had unlabeled LLM-authored commits at some point before 2026. Looking more closely at rsync's recent commit history, I think it's less likely in this case. There's just a low number of commits in general, _until_ large batches of Claude-authored commits start showing up early this year. But this then raises some questions about the bugs-per-commit metric; it does correct for something like "size of release", but also obscures a significant shift in commit velocity that may be downstream of adding LLM development tools to the workflow.

Like I said, I don't have a dog in this fight, and I try not to approach sorts of questions from a position of explicit advocacy. I do think it's an interesting question, though, and we should try to understand what the data is actually telling us.

Re: Did Claude increase bugs in rsync?

#119

Post by hypfer »

logicprog wrote: post #116
hypfer wrote: post #76

Earlier quoted context omitted.

> I decided to rewrite all of the prose in my own voice. Soo... it didn't just sound like genai but was genai? ___ Huh. From the article: > If anyone complains about my verbosity or sentence structure — as they usually do, which is the reason I originally let the AI write the prose, among other reasons obsoleted by templating — they can go fuck themselves. This is kinda sad, honestly. But also should show the author…

> .. what are those em-dashes doing there though? You're literally doing exactly the bullying I was trying to avoid, even while denouncing it. I like em-dashes . I have AuDHD, and they help me represent how I think.

[flagged]

Re: Did Claude increase bugs in rsync?

#120

Post by skeledrew »

jchw wrote: post #63
skeledrew wrote: post #55

Earlier quoted context omitted.

I read it perfectly fine. I see content, not style.

When you say, "I see content, not style," you are separating what is being said from how it is being said. While it is great that you can extract the core message, you are missing a fundamental truth about writing: style and content are rarely completely separate. Writing involves both. Poor prose does not just make writing ugly — it creates friction, obscures nuance, and introduces ambiguity. You can eat a gourmet m…

The internet is going to really suck for you if you keep that attitude, because LLM use will only increase. Though also maybe not too much as the LLM-isms will likely be fine-tuned out of them to the point that the only way you'll be sure something is done with one is if the author left a note saying such. But maybe that'll make it suck even more as then you'd be without a definite target most of the time, always wondering how much of the thing you're reading is by human and how much by LLM...
Post reply on HN