Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Did Claude increase bugs in rsync?

alexispurslane.github.io

Post reply on HN
Print view View original

431–440 of 611 posts

Re: Did Claude increase bugs in rsync?

#431

Post by kelnos »

Eufrat wrote: post #408
tpm wrote: post #405

Earlier quoted context omitted.

> Selling a toaster has an implicit warranty of merchantability. Why would you think this is worth mentioning here? Instead of explaining, just try to do something, that people actually use, for free, in the open, for some time. It doesn't have to be software, can be work for a nonprofit or a charity etc. I'm sure you will be enlightened.

I volunteer and I don't tell people or believe they should be grateful that an event is happening because of the volunteers. I just don't find this logic compelling in the same way that you don't find my logic compelling either.

I'm a little confused at this comment, because the person you're replying to didn't say anything to the effect of what you're arguing against here.

Re: Did Claude increase bugs in rsync?

#432

Post by kelnos »

wolletd wrote: post #420
Eufrat wrote: post #327

Earlier quoted context omitted.

I think Tridge is simultaneously trying to be proactive and kinda giving too much credit to marketing. Anthropic has not been able to really give numbers or actual values on what Mythos can really do. It just waved Mythos in front of the public like a boogeyman screaming that AI is going to cause a security nightmare (and it has, but mostly through vibe coded trash from what I’ve noticed); I’m hard pressed to find th…

> “I’d rather be out sailing than working on rsync security issues, so I have reached for several AI tools to help with what needs to be done,” Well, then maybe it's already overdue to find a new maintainer for the project and let someone else continue it? The tool will not get better from someone working on it who doesn't want to.

Unless you're willing to step up and be that person, it's not your place for you to suggest it.

Re: Did Claude increase bugs in rsync?

#433

Post by logicprog »

xmddmx wrote: post #338

There's a meta-level of irony here that's important to note. TFA is defending the use of AI, and it very clearly (to me) used AI to analyze the data and present the results. In doing so, the author used statistics in a way they do not appear to understand, and ended up making numerous false claims (you can see the thread discussing these here https://news.ycombinator.com/item?id=48417626 ) In short, the study doesn't…

The statistical methodology I used is mine. As is the interpretation. Completely. To the degree that I misunderstood statistics (and it is under debate even in the thread you link, and the people accusing me of misunderstanding statistics there are universally misrepresenting my point, which is to point out a total absence of evidence for any difference, not to prove the null hypothesis) that's on me

Re: Did Claude increase bugs in rsync?

#434

Post by logicprog »

cobertos wrote: post #388
logicprog wrote: post #280

Earlier quoted context omitted.

> Why was v3.4.1 the most buggy, right before the Claude commits? Why did "nobody notice"? It's way to strange to just say welp, it must be human error. Why wouldn't it be except question begging priors assuming it couldn't be? > Why does v3.4.2 have 0 bugs, or 0 bug score. And why was such an outlier (no other commit seemingly has this??) allowed to mix into aggregate statistics and bring all the "is Claude buggy?"…

> Why wouldn't it be except question begging priors assuming it couldn't be? A multitude of reasons? A change in maintainer. A change in the mental state of a maintainer. A sudden focus by the community on a given undesirable behavior. Someone else here suggested use of Claude AI before it was disclosured. The framing implies that it was human-produced coding error, but my point is it could be _any other human error_…

I'll do that when I get a chance

Re: Did Claude increase bugs in rsync?

#435

Post by kelnos »

ekidd wrote: post #358
rendaw wrote: post #347

Earlier quoted context omitted.

Is using calloc for everything fixing a security issue or hardening it?

Calloc is generally hardening, because it zeros out any stale memory contents left over from previous uses of the memory. You can avoid this overhead if you use a language that forbids reading from uninitialized memory, but C is not that language.

Uninitialized memory is not a problem (the OS is never going to give a program memory that has data in it from another program). The problem is memory that you allocated in the past, have freed, but hasn't been returned to the OS[0]. It might have key material or other sensitive data in it[1]. Or it might just have random garbage in it that could be misinterpreted by the code that's about to use it, if it hasn't been initialized to a known state.

For some uses, you do genuinely need (specifically) zeroed-out memory before you start to use it, and that's where calloc() is truly useful. But that need not have anything to do with security.

[0] The allocator will often hold onto memory that has been freed in order to quickly service future requests for new allocations, without needing a context switch into kernel space.

[1] Granted, the correct way to handle that is to zero it out before freeing it, in a way that the compiler won't optimize out.

Re: Did Claude increase bugs in rsync?

#436

Post by cthalupa »

Barrin92 wrote: post #427
JetSetIlly wrote: post #422

Earlier quoted context omitted.

> This is not okay. Agreed. The way to address it though, is through calm analysis and reason. The emotional language from both groups is not helping. If there's one problem with Claude et al, it's that it's all happened way too quickly for people to keep up. We're all at different stages of acceptance and I think that's what we're seeing manifest in the various discussions.

>We're all at different stages of acceptance I do hope you see the irony of accusing people of armchair psychology and then hitting us with the five stages of grief. I trust rsync (which handles critical data on my system) because I know a veteran of 40 years wrote the code it runs. If I see code like the one above posted by the OP, that the author wouldn't have written, I start to pay attention. When I then read the…

> If I see code like the one above posted by the OP, that the author wouldn't have written, I start to pay attention.

Except the author did write it. https://github.com/RsyncProject/rsync/issues/959#issuecommen...

Which is part of the problem with all of this nonsense right now - everyone is running off of emotion and not looking to see if what is being said is actually true. Which is somewhat ironic, considering the message of the article.

Re: Did Claude increase bugs in rsync?

#437

Post by [deleted] »

jagged-chisel wrote: post #262
delusional wrote: post #248

Earlier quoted context omitted.

You don't really have to guess. The guy told us the AI didn't suggest this specific change: > The change to zero memory was my idea and my change. It was a reaction to a security report I got which caused use of an element past the end of an array. By zeroing the allocation I could ensure that misuse of that memory if a similar bug came up in the future could only cause a null ptr deref, which is better than the chan…

> … By zeroing the allocation … How does that prevent reading past the end of the buffer? Or change how bytes outside the buffer are used? Are these arrays of pointers so that the “null ptr deref” comment makes sense? Or am I the bozo and don’t know what’s happening here?

[deleted]

Re: Did Claude increase bugs in rsync?

#438

Post by alfiedotwtf »

GodelNumbering wrote: post #234

Was just looking at commits and came across a commit and its revert original commit: https://github.com/RsyncProject/rsync/commit/d046525de39315d... ``` - if (!ptr) - ptr = malloc(num * size); - else if (ptr == do_calloc) + if (!ptr || ptr == do_calloc) ptr = calloc(num, size); ``` Written with claude. This is a good example of what slips through LLM attention. It forces all allocations to be calloc as if it is a str…

AI is fine, and in fact fun to use... committing AI written code without understanding Every. Single. Line. Of. Changes is on the committer. You can't LGFM for vibe code ffs

Re: Did Claude increase bugs in rsync?

#439

Post by logicprog »

bsza wrote: post #418
logicprog wrote: post #140

Earlier quoted context omitted.

I've now resolved this. The new version, which should be live on GH Pages soon, uses — what I think is — a pretty good methodology for assigning severity to each bug, normalizes it to 0.0-1.0, sums that, and treats that as the total severity weighted bugs, then does the analysis based on that. It did not change the analysis in any material way.

No Claude, it still makes zero sense as a metric. A commit is a measure of nothing. Severity weighted bugs per unit of nothing? What does that even mean? In any repo it's trivial to achieve a sev/10c that's arbitrarily close to zero while completely ruining everything. I suggest you practice some humility and update your conclusion instead of updating the mental gymnastics you used to arrive at the same conclusion.

Whether commits decrease the sev/10c depends on if there are a lot of small commits increasing the demoninator. In reality, we have the opposite: the post-Claude releases have way fewer commits than the pre-Claude ones.

Thus, if anything their sev/10c is inflated. If I changed it to lines of code changed, the relative bug ratios would be much smaller, and the conclusion wouldn't change. In fact, the conclusion would look "better" for Claude; if I was using "mental gymnastics" to come to this conclusion, I would have already used a metric other than adjusting per commits!

What different metric would you suggest that would change the conclusion?

Showing "humility", as you so moralistically and condescendingly put it, would require being wrong first.

Post reply on HN