Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Did Claude increase bugs in rsync?

alexispurslane.github.io

Post reply on HN
Print view View original

241–250 of 611 posts

Re: Did Claude increase bugs in rsync?

#241

Post by fragmede »

block_dagger wrote: post #197
Aurornis wrote: post #135

Earlier quoted context omitted.

The value of the Claude attribution is that you can tell at a glance who used AI. I don't care about the advertising angle. We all know Claude by now. I want some indicator that AI was used.

At my employer, if AI is not used, it shows up on your performance report and you’ll be told if you don’t start using it, you will be dismissed. I work at a medium sized successful YC-backed SaaS. So here, the attribution is meaningless - they look at your Bedrock and LLM API calls as well as Claude Code history.

Do you fellow ICs have access to those reports and can correlate commits from you to the prompts used to create them easily?

Re: Did Claude increase bugs in rsync?

#242

Post by Aurornis »

block_dagger wrote: post #197
Aurornis wrote: post #135

Earlier quoted context omitted.

The value of the Claude attribution is that you can tell at a glance who used AI. I don't care about the advertising angle. We all know Claude by now. I want some indicator that AI was used.

At my employer, if AI is not used, it shows up on your performance report and you’ll be told if you don’t start using it, you will be dismissed. I work at a medium sized successful YC-backed SaaS. So here, the attribution is meaningless - they look at your Bedrock and LLM API calls as well as Claude Code history.

If the company policy is to have everyone using it then everyone is going to assume you're using it.

I don't see a need for an attribution line in this case.

Re: Did Claude increase bugs in rsync?

#243

Post by wlonkly »

lbrito wrote: post #181

Wait, how is any of this relevant if there were only 2 Claude commits? My statistics courses are far behind me, but don't you need at least 30 data points to conclude anything?

It's not uncommon to have small amounts of data come out of experiments. These are appropriate tests for the size of the data. These tests failed to disprove the null hypothesis.

Re: Did Claude increase bugs in rsync?

#244

Post by archagon »

potsandpans wrote: post #216
eschaton wrote: post #207

Earlier quoted context omitted.

That’s not the only thing that matters. The provenance of the code also matters enormously, specifically whether the person contributing it actually has the right to do so. If I contributed code to an Open Source project behind my old employer’s back, that would have been bad, because that code was owned by them and not me, even if I wrote it on my own time using my own equipment, because of the contract I signed wit…

[flagged]

Is this comment LLM generated?

Have fun with 1000x more Buns that literally no one is using or maintaining. An entire software industry built on top of a burning garbage pile of crappy, dead code.

Re: Did Claude increase bugs in rsync?

#246

Post by wolletd »

GodelNumbering wrote: post #234

Was just looking at commits and came across a commit and its revert original commit: https://github.com/RsyncProject/rsync/commit/d046525de39315d... ``` - if (!ptr) - ptr = malloc(num * size); - else if (ptr == do_calloc) + if (!ptr || ptr == do_calloc) ptr = calloc(num, size); ``` Written with claude. This is a good example of what slips through LLM attention. It forces all allocations to be calloc as if it is a str…

Also the amount of commits is suspicious. In the last two months, rsync had about as much commits as in the last two years before that. Most of them written with claude. And then stuff like this is in there.

That's exactly what I'd expect when someone is excited about AI usage and becomes... well, sloppy.

Re: Did Claude increase bugs in rsync?

#247

Post by scottlamb »

GodelNumbering wrote: post #234

Was just looking at commits and came across a commit and its revert original commit: https://github.com/RsyncProject/rsync/commit/d046525de39315d... ``` - if (!ptr) - ptr = malloc(num * size); - else if (ptr == do_calloc) + if (!ptr || ptr == do_calloc) ptr = calloc(num, size); ``` Written with claude. This is a good example of what slips through LLM attention. It forces all allocations to be calloc as if it is a str…

> This is a good example of what slips through LLM attention. It forces all allocations to be calloc as if it is a strict upgrade.

I wouldn't assume Claude made that decision; it's not as if that was some incidental thing that it snuck into a large commit. The commit message starts with "zero all new memory from allocations", and that's exactly what the commit does. What do you imagine the prompt was?

It seems totally plausible to me that a human initially thought this was an improvement, then rethought after discovering the RSS regression. And it's not a law of nature anyway that this change has to increase RSS; calloc could special-case the case in which memory was freshly returned from the OS, knowing fresh memory mappings are zeroed anyway.

I blame AI for these regressions mostly in the sense that it caused a flurry of vulnerability reports. Those led to a flurry of quick fixes. Sometimes quick fixes cause other problems.

Re: Did Claude increase bugs in rsync?

#248

Post by delusional »

scottlamb wrote: post #247
GodelNumbering wrote: post #234

Was just looking at commits and came across a commit and its revert original commit: https://github.com/RsyncProject/rsync/commit/d046525de39315d... ``` - if (!ptr) - ptr = malloc(num * size); - else if (ptr == do_calloc) + if (!ptr || ptr == do_calloc) ptr = calloc(num, size); ``` Written with claude. This is a good example of what slips through LLM attention. It forces all allocations to be calloc as if it is a str…

> This is a good example of what slips through LLM attention. It forces all allocations to be calloc as if it is a strict upgrade. I wouldn't assume Claude made that decision; it's not as if that was some incidental thing that it snuck into a large commit. The commit message starts with "zero all new memory from allocations", and that's exactly what the commit does. What do you imagine the prompt was? It seems totall…

You don't really have to guess. The guy told us the AI didn't suggest this specific change:

> The change to zero memory was my idea and my change. It was a reaction to a security report I got which caused use of an element past the end of an array. By zeroing the allocation I could ensure that misuse of that memory if a similar bug came up in the future could only cause a null ptr deref, which is better than the chance of a valid pointer. It got a claude co-authored tag on it as I got it to do some tidy ups of a series of commits, and that is just what it does when it makes any modification. It doesn't mean the change was written by claude. It was written by me.

https://github.com/RsyncProject/rsync/issues/959#issuecommen...

Re: Did Claude increase bugs in rsync?

#249

Post by iainctduncan »

What strikes me about the post is that it goes to great lengths to talk about proper statistical methods, but then is written in the most clearly biased language ("what stupid AI haters get wrong etc). If you want people to take your study seriously, why wreck it by coming across with such a strong prior bias? I stopped reading...

Re: Did Claude increase bugs in rsync?

#250

Post by matheusmoreira »

toofy wrote: post #239
matheusmoreira wrote: post #143

Earlier quoted context omitted.

And why do you want to know that? So you can call our projects slop? Ostracize us?

for the same reason we want to know who wrote an article, a book, a movie, a song, a play, a journal paper, a painting, and on and on. why do you so many people want to hide who the real author is? we should be very weary of anyone claiming they’re the author of something when they’re absolutely not. if jon wrote a book and i take credit, that’s shady as hell.

Ghostwriting is a thing.
Post reply on HN