Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Did Claude increase bugs in rsync?

alexispurslane.github.io

Post reply on HN
Print view View original

121–130 of 611 posts

Re: Did Claude increase bugs in rsync?

#121

Post by logicprog »

hypfer wrote: post #90
ajkjk wrote: post #85

Earlier quoted context omitted.

The em dashes are fine. If someone gives them shit about their writing, that's on the critic for being shitty. If they use AI to write, that's on them for being fake. But, to write online at all requires being ready to have people be shitty to you and ideally not reacting in a way that makes the situation worse. Sounds like they need work on that part. Anyway it is basically always possible for someone to find someth…

I don't think that you can successfully dismiss an obvious AI writing marker with "No these are fine, now look over there!! " Pay no attention to the man behind the curtain?

Great, so I rewrite everything in my own prose, and now it's still "obvious AI writing," just because I'm literate.

Re: Did Claude increase bugs in rsync?

#122

Post by mikaeluman »

Not going to critique this survey. Must have taken a lot of time and required a lot of patience. Great work!

I think it will be up to some group in academia to make a real full blown study across several repositories.

There must be tons to learn on how LLMs have changed software development and perhaps the cleanest separation will simply be going by what repositories declare e.g. "No LLM involved" vs those that proudly do the opposite or are neutral.

Bugs is not the only variable of interest here. I am guessing someone is already doing this as we discuss it here...

Re: Did Claude increase bugs in rsync?

#123

Post by zzyzxd »

thorum wrote: post #80

Unfortunately for the people mad about this, I predict the only thing they will accomplish by pressuring the rsync maintainers, is to discourage everyone else from responsibly disclosing their use of AI. You’re just going to make people disable Claude attribution on their commits to avoid drama.

I never care about AI usage disclosure, because I don't believe that human produced code is necessarily better than AI produced code, unless it's someone I personally know.

People need to be responsible for code they commit and push anyways. This has never changed. Whether the code is written by hand, by their cat walking over keyboard, or by AI, is not my concern.

A project's code quality can decline for all kinds of reasons. I don't think it's productive to laser-focus on whether it's produced by AI or not. That's a distraction. If a person just want to find excuse to criticize AI, and another person wants to fight back and defend AI, sure, go for it. But that's not how you would want to assess a project's code quality.

Re: Did Claude increase bugs in rsync?

#124

Post by skeledrew »

grey-area wrote: post #87
skeledrew wrote: post #55

Earlier quoted context omitted.

I read it perfectly fine. I see content, not style.

Style is also part of the content. Word choice, grammar, register, and tone all affect meaning and communication of that meaning. The medium is part of the message. So your statement betrays a significant misunderstanding - there is no neat clean divide between style and content. Also, LLMs often generate text that is plausible, but wrong, in ways big and small.

Well, I got the meaning in the article fine, and have no complaints.

> Also, LLMs often generate text that is plausible, but wrong, in ways big and small.

So do humans. Always have, always will.

Re: Did Claude increase bugs in rsync?

#126

Post by overgard »

nairboon wrote: post #11

Is this an analysis made by/with Claude?

FWIW, I asked ChatGPT to review the article just for my amusement. It's conclusion was:

"My honest assessment is that this is a competent calculation performed on a badly confounded measurement, followed by conclusions substantially stronger than the calculation warrants. It is useful as a rebuttal to “the Claude releases are obviously unprecedented disasters,” but not as evidence that Claude was harmless."

Re: Did Claude increase bugs in rsync?

#127

Post by logicprog »

Another update: did an automated severity analysis on each bug report (~2000 of them!) using an LLM at temp=0 with a very strict rubric (and I checked to make sure that it rated things in a consistent, stable way using it). The rubric, LLM used, and some example ratings are included in the methodology section. For now, the information was just stored per-bug in the DuckDB and used to filter out non-bug bugs, to get a clearer signal. I'm going to try to use it to see if the post-Claude bugs were more severe in any way next.

Re: Did Claude increase bugs in rsync?

#128

Post by AlienRobot »

amiga386 wrote: post #115
julianeon wrote: post #109

Earlier quoted context omitted.

If Claude is actually good enough to commit to rsync, of course I'm going to look at that and think "it's good enough for my side project too." And (benefit to companies aside) that is info it is useful to know, if it's true.

Yeah, this is why it's obnoxious and this is why scummy marketers do it. If you don't aggressively turn it off, they leech an implicit endorsement out of you. - Sent from my iPhone

Indeed. The best endorsement is done explicitly by obnoxious users.

I use Linux, btw.

Re: Did Claude increase bugs in rsync?

#129

Post by redsocksfan45 »

matheusmoreira wrote: post #104
thorum wrote: post #80

Unfortunately for the people mad about this, I predict the only thing they will accomplish by pressuring the rsync maintainers, is to discourage everyone else from responsibly disclosing their use of AI. You’re just going to make people disable Claude attribution on their commits to avoid drama.

> You’re just going to make people disable Claude attribution on their commits to avoid drama. People should be doing this regardless of drama. No reason to provide free advertising for trillion dollar corporations. Generated-by trailers are only relevant when contributing to third party projects, in that case disclosure is polite.

[dead]

Re: Did Claude increase bugs in rsync?

#130

Post by runarberg »

logicprog wrote: post #108
floxy wrote: post #98

Earlier quoted context omitted.

Seems like this would be a good place to link to that.

I link to it multiple times in TFA and quote the specific thing I'm talking about here in there to explain that possible confounder. I think I've done more than the work I'm obligated to it.do to make all of the relevant information available to you. You are just refusing to use

I am not finding these links in TFA, I see a link to an issue #929 which (as mentioned in TFA) has over 350 replies, and and opinionated summary of what transpired, including some detailed description of specific posts there. However I did not find the maintainers response.

Of interest is this post here: https://github.com/RsyncProject/rsync/issues/929#issuecommen... which echos the same concern which was raised up thread, however, I failed to find the maintainers’ response.

EDIT: Found it! it is in the (untitled) discussion section (after the results).

https://lobste.rs/s/k1b0za/rsync_outrage#c_2iowov

EDIT 2 (and advice on design): The page design changes backgrounds after the results sections, which kind of conveys to the user that they have reached the end of what was is important and can just skim over the rest (usually pages have a radical change in typography like these when you’ve reached the comment section), however this is what is analogous to a discussion in a typical paper, and is arguably the most important part. I had simply assumed that you just left it at the result and skipped the discussion as a stylistic choice.

Post reply on HN