Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Did Claude increase bugs in rsync?

alexispurslane.github.io

Post reply on HN
Print view View original

351–360 of 611 posts

Re: Did Claude increase bugs in rsync?

#351

Post by davrosthedalek »

nerevarthelame wrote: post #326
iandinwoodie wrote: post #302

Earlier quoted context omitted.

Also from the article: > "Claude clearly made things worse" &emdash; the main claim This article was clearly generated by AI, yet I found no mention/attribution of that by author. How likely is it than someone who vibe codes articles would also vibe code the underlying analysis and be eager to accept an outcome that is highly validating of that person’s workflow? I’d say very.

He did admit as much: > "The scripts used to fetch the data, collate it into a DuckDB database file, construct the views on that DB, and then do the statistical analysis on that data, were indeed written by GLM 5.1, as was the HTML and much of the original prose for the final report webpage you're looking at right now."

But: "After posting this on Hacker News and recieving [sic] almost no substantive input, discussion, or response on the actual content of the article, I decided to rewrite all of the prose in my own voice. If anyone complains about my verbosity or sentence structure — as they usually do, which is the reason I originally let the AI write the prose, among other reasons obsoleted by templating — they can go fuck themselves."

So rewritten in his own voice. Maybe the m-dashes are from GLM, maybe from the author.

Re: Did Claude increase bugs in rsync?

#352

Post by tom_ »

nomel wrote: post #264
jarym wrote: post #256

I've been coding for over 2 decades. I love it, I've always loved it and I likely always will. I was an AI skeptic some months ago but truly Claude and Codex have changed my development style and velocity in a way I never imagined would ever be possible. With that, yes, I produce more code and am finding more bugs. So looking over at comments in HN articles the amount of polarising hate to anything produced with AI i…

I've always noticed, within any subject involving tools, there are people who like the tools, and some people who like to use the tools to do something else. With programming, I've always been in the later: it's a tool that allows me to do what I actually love, which is problem solving, system level thinking , and providing some nice solution to that problem, that happens to be through software. So, I have an absolut…

I just really hate talking to the computer in human language.

Re: Did Claude increase bugs in rsync?

#353

Post by Ronsenshi »

int_19h wrote: post #319
eschaton wrote: post #221

Earlier quoted context omitted.

Yes.

You're not entitled to know what specific tools were used to produce something, generally speaking. In the absence of such an entitlement, not volunteering to disclose the tools used is not fraud.

Don't think calling a PR written by AI is the same thing as using a "tool". If code is largely generated by AI means that AI was an author and not you with some tool.

Re: Did Claude increase bugs in rsync?

#354

Post by rsc »

Eufrat wrote: post #327
logicprog wrote: post #257

Earlier quoted context omitted.

Tridge already explains this: "Like many developers of open source packages I’ve been hit by a flood of security reports lately in my role as the rsync maintainer. Many of those reports are AI generated (not all though, there are some notable ones with very careful and high quality manual analysis). As this flood started to get more intense I realised I needed to raise the defences on rsync a lot — we needed much mor…

I think Tridge is simultaneously trying to be proactive and kinda giving too much credit to marketing. Anthropic has not been able to really give numbers or actual values on what Mythos can really do. It just waved Mythos in front of the public like a boogeyman screaming that AI is going to cause a security nightmare (and it has, but mostly through vibe coded trash from what I’ve noticed); I’m hard pressed to find th…

As a long-time open-source maintainer, I find all the second-guessing and armchair psychoanalysis here (not just in this comment, all over HN) about Tridge's motivations, state of mind, and so on incredibly off-putting.

Tridge doesn't owe anyone anything as far as rsync is concerned. Yet he is spending his time maintaining it, only to be attacked for his efforts.

To respond to the specific technical point, there really _is_ a flood of security reports arriving everywhere in the past few months. The jury is out on whether Mythos is that much better than alternatives, but even the publicly available models are _highly_ capable of finding real problems, and they are being employed to that end quite effectively. Here are the counts of security issues fixed in each monthly Go minor release going back to the start of 2024:

     0 2024-01-09 Go 1.21.6, Go 1.20.13
     0 2024-02-06 Go 1.21.7, Go 1.20.14
     5 2024-03-05 Go 1.22.1, Go 1.21.8
     1 2024-04-03 Go 1.22.2, Go 1.21.9
     2 2024-05-07 Go 1.22.3, Go 1.21.10
     2 2024-06-04 Go 1.22.4, Go 1.21.11
     1 2024-07-02 Go 1.22.5, Go 1.21.12
     0 2024-08-06 Go 1.22.6, Go 1.21.13
     3 2024-09-05 Go 1.23.1, Go 1.22.7
     0 2024-10-01 Go 1.23.2, Go 1.22.8
     0 2024-11-06 Go 1.23.3, Go 1.22.9
     0 2024-12-03 Go 1.23.4, Go 1.22.10
     
     2 2025-01-16 Go 1.23.5, Go 1.22.11
     1 2025-02-04 Go 1.23.6, Go 1.22.12
     1 2025-03-04 Go 1.24.1, Go 1.23.7
     1 2025-04-01 Go 1.24.2, Go 1.23.8
     1 2025-05-06 Go 1.24.3, Go 1.23.9
     3 2025-06-05 Go 1.24.4, Go 1.23.10
     1 2025-07-08 Go 1.24.5, Go 1.23.11
     2 2025-08-06 Go 1.24.6, Go 1.23.12
     1 2025-09-03 Go 1.25.1, Go 1.24.7
    10 2025-10-07 Go 1.25.2, Go 1.24.8
     * 2025-10-13 Go 1.25.3, Go 1.24.9
     0 2025-11-05 Go 1.25.4, Go 1.24.10
     2 2025-12-02 Go 1.25.5, Go 1.24.11
    
     6 2026-01-15 Go 1.25.6, Go 1.24.12
     2 2026-02-04 Go 1.25.7, Go 1.24.13
     5 2026-03-05 Go 1.26.1, Go 1.25.8
    10 2026-04-07 Go 1.26.2, Go 1.25.9
    11 2026-05-07 Go 1.26.3, Go 1.25.10
     3 2026-06-02 Go 1.26.4, Go 1.25.11
* The Go 1.25.3 and Go 1.24.9 releases were a fast follow to fix a problem introduced by one of the security fixes the previous week.

You can see that 2026 has been quite different from the previous years. There are plenty of other contemporaneous accounts from other security teams about the load increase they've seen (which again is almost entirely not Mythos).

Also, the number of reports we are receiving has gone up far faster than the number of actual vulnerabilities. Over the 75-month period from January 2020 to early April 2026, the final 30 days accounted for ~16% of the reports.

It is easy to believe that Tridge is seeing a similar flood of reports. More reports means more fixes means more code changes means more bugs.

Re: Did Claude increase bugs in rsync?

#355

Post by archagon »

int_19h wrote: post #341
archagon wrote: post #323

Earlier quoted context omitted.

Things have been bad, but every company using its own bespoke LLM reimplementation of rsync and similar is so, so much worse.

Why would every company do it though? They'll just all be using the same (Anthropic's) AI-enabled fork.

You think Anthropic wants to be the sole maintainer of thousands of forked OSS projects...? I seriously doubt that would happen, for legal, marketing, and logistical reasons alike.

Re: Did Claude increase bugs in rsync?

#356

Post by saagarjha »

int_19h wrote: post #331
saagarjha wrote: post #324

Earlier quoted context omitted.

Because Fil-C is not a serious project

If you make claims like that, you need to expand on them or at least provide some references.

It’s Fil’s side project that he uses to spend his extra creative energy and troll people on Twitter

Re: Did Claude increase bugs in rsync?

#357

Post by runarberg »

logicprog wrote: post #215
runarberg wrote: post #195

Earlier quoted context omitted.

You are interpreting the p-values on their own merit rather then using them to test a null-hypothesis. Quotes like: > With a p-value of 74%, the answer is a decisive no. The odds ratio is 1.06 — essentially 1:1. Claude releases are no more likely to be above the median than any other releases. are problematic in this context as the correct conclusion here is you just don‘t have enough data conclude whether or not you…

Edited that claim, and made several clarifications elsewhere. The whole point of this analysis is that outrage is unjustified on the basis of two totally statistically unremarkable releases that no one would have remarked on pre-AI (my further proof of this is that there was a pre-AI remarkably broken release, and no one did comment!) and zero positive evidence outside cherry-picked anecdotes for any negative impact.…

There is a lot more context to the outrage which is missing from your analysis. People have multiple reasons to be mad at AI usage, you mention some of them in your introduction, and you put a (statistically insignificant) measure on only one of them. In your analysis you have shown that exactly one of these reasons is anecdotal. That does not mean they are wrong, and it especially does not mean they are unjustified.

That you found a single pre-AI release which did not cause outrage is proof of nothing. This single release is equally anecdotal, and statistically insignificant.

So, the biggest context that is missing here is that people hate AI for various reasons, and they don‘t want their favorite tools to fall victim to AI for equally many reasons. It is only natural that people who hate AI react this way when they find out their favorite tool uses AI, and doubly so when they sniff correlation between their favorite tools use of AI and bugs.

> I'm just trying to say that these specific releases are unremarkable, and there's no evidence at all of harm currently.

Well, there is no evidence against harm either. But what you did here is a bit of a slight of hand. In your analysis your null hypothesis is: “There is no difference in bug count between releases which includes code commits from Claude Code and releases which don‘t”. (You then go about doing what every psychology major is taught not to do; find evidence for the null hypothesis, not against it). However what hypothesis testing is for is to use a representative sample to generalize over a wider population. You do hypothesis testing because you want to demonstrate that your sample is representative of a wider population, that you just so happened to have picked the two sample, by random chance, which shows the effect regardless of the experiment.

By calculating the p-values you were telling me that you were in fact ready to make generalizing statements over a wider population of commits, but your results were statically insignificant, so really you should not draw any conclusions from them. You have not, in fact, shown that they aren’t different from the rest of the population.

Re: Did Claude increase bugs in rsync?

#358

Post by ekidd »

rendaw wrote: post #347
logicprog wrote: post #257

Earlier quoted context omitted.

Tridge already explains this: "Like many developers of open source packages I’ve been hit by a flood of security reports lately in my role as the rsync maintainer. Many of those reports are AI generated (not all though, there are some notable ones with very careful and high quality manual analysis). As this flood started to get more intense I realised I needed to raise the defences on rsync a lot — we needed much mor…

Is using calloc for everything fixing a security issue or hardening it?

Calloc is generally hardening, because it zeros out any stale memory contents left over from previous uses of the memory.

You can avoid this overhead if you use a language that forbids reading from uninitialized memory, but C is not that language.

Re: Did Claude increase bugs in rsync?

#359

Post by albedoa »

jiggawatts wrote: post #273
jarym wrote: post #256

I've been coding for over 2 decades. I love it, I've always loved it and I likely always will. I was an AI skeptic some months ago but truly Claude and Codex have changed my development style and velocity in a way I never imagined would ever be possible. With that, yes, I produce more code and am finding more bugs. So looking over at comments in HN articles the amount of polarising hate to anything produced with AI i…

I work with outsourced code all the time and it is a tyre fire without exception. I just spent a week scrubbing a codebase where some dev “did the needful” and committed an on-by-default flag to bypass authentication checks because he didn’t known how to set up his local work environment. People report the same “took a shortcut” issue with AI vibe coding, and I can confirm that I’ve had to rewrite practically everyth…

> I work with outsourced code all the time and it is a tyre fire without exception. I just spent a week scrubbing a codebase where some dev “did the needful” and committed an on-by-default flag to bypass authentication checks because he didn’t known how to set up his local work environment.

OP knows this but finds himself in the strange position of having to defend India slop in order to defend AI slop, totally unnecessarily and unprompted. It's baffling to you and me.

Post reply on HN