Wait, how is any of this relevant if there were only 2 Claude commits? My statistics courses are far behind me, but don't you need at least 30 data points to conclude anything?
Did Claude increase bugs in rsync?
181–190 of 611 posts
Re: Did Claude increase bugs in rsync?
#182The fact last few commits were attributed to claude doesn't mean previous ones didn't use it. Also if you write a paper where you get statistical conclusions out of whole 2 datapoints you'd be laughed out of the room
Why not? Claude marks its commit messages. That there were none, and then there were, seems a signal.
Especially since if the earlier commits were so clearly AI authored yet without the Claude marker, surely you or anyone would be able to spot them. You could say, X commit does not have the Claude commit marker yet was AI written. But for all the speculation on this thread, I haven’t seen anyone actually doing that. What may be possible is that the rsync maintainers used AI to assist yet reviewed and edited themselves, as many devs do, and if so then the stats in this article are still notable: there are no poor quality outliers that can reliably be attributed to AI and if one specific release (3.4.0) was, the subsequent releases which presumably also had as much AI as this speculative hidden AI release only show improvement and thus act as a pro-AI argument.
The blog has many more datapoints than two. It compares many releases. You’re looking at 2-vs, not 2.
Re: Did Claude increase bugs in rsync?
#183This is just narrow thinking. Say Claude did increase the bugs in rsync by a negligible factor. So what? You've saved a significant amount of time for a decent number of humans, and if those humans are working on other projects, the overall net output for the world is net positive compared to without LLMs. You have to broaden your perspective. It's not just about how rsync was affected.
Let me translate this comment:
> ok, so I was wrong and badly, but I will double down and say I was right anyway
Re: Did Claude increase bugs in rsync?
#184Earlier quoted context omitted.
The value of the Claude attribution is that you can tell at a glance who used AI. I don't care about the advertising angle. We all know Claude by now. I want some indicator that AI was used.
And why do you want to know that? So you can call our projects slop? Ostracize us?
Some people prefer organic grown food for all kinds of reasons, does it matter to you they would want the same for code? (Also, I'm not picking a side here)
Re: Did Claude increase bugs in rsync?
#185I think there's evolution at play here - if you dislike AI enough to opt out of using any ai-generated code, you will likely suffer. I think there's definitely a conversation to be had about whether to disclose AI use or not but that's a separate issue if you assume that everyone is using it in some respect.
Re: Did Claude increase bugs in rsync?
#186Wait, how is any of this relevant if there were only 2 Claude commits? My statistics courses are far behind me, but don't you need at least 30 data points to conclude anything?
Depends on the methods you use. If you're trying to fit curves and so on, yes. The methods I use were designed for very low amounts of data, and are generally okay for that, specifically and especially when you're just trying to show a lack of evidence for some non-null hypothesis.
And again, that's kind of the point. There's exactly zero actual evidence, however you slice it, that "Claude broke rsync" except cherry-picked anecdata, and the whole point of my analysis is to demonstrate the total lack of any such trend/evidence at all, and just how in-distribution/normal these releases are, to show that if people hadn't known Claude was involved in them, they wouldn't have remarked on them.
Re: Did Claude increase bugs in rsync?
#187Earlier quoted context omitted.
And why do you want to know that? So you can call our projects slop? Ostracize us?
Some people prefer organic grown food for all kinds of reasons, does it matter to you they would want the same for code? (Also, I'm not picking a side here)
It matters when I'm contributing to their projects. In that case I'll go out of my way to be polite and learn their rules.
Re: Did Claude increase bugs in rsync?
#188Earlier quoted context omitted.
Let's start with most outright alarming error - the claude statistics are taken out of whole 2 data points
That's sort of the point. There isn't enough data to extrapolate, and yet that's exactly what those outraged about AI were doing, and when you do do the very minimal types of analyses (permutation tests, and looking at distributions, mostly) that are actually valid, safe, standard, and useful to do on such low amounts of date, again, no evidence for the outrage shows up, and the two releases look so normal that it so…
I know you’re defending your work here but this behavior does absolutely nothing to help your point.
Re: Did Claude increase bugs in rsync?
#189Earlier quoted context omitted.
> which will undergo human review This can be largely assumed to be true for any open source code. It's kinda the point of open source.
Nope. It cannot be assumed at all. Maintainer could just as easily tell Claude to review the hand written code you sent instead of spending any effort on it. Maintainer could sit on the patch for months on end only to swoop in later and rewrite it instead of engaging with you, thereby erasing your contribution and attribution. Maintainer could just ignore you entirely despite the pervasive "patches welcome" attitude.…
I'm referring to the fact that "open source" quite literally means "readable by humans [and machines]", and anything beyond that is a subject of debate. There are more users than readers in nearly all cases, but being able to read the code as a user is a significant benefit at times, and it's one of the reasons it's such a large ecosystem in terms of both users and contributors. (it usually being free is another big reason, of course)
Even with coding agents gaining popularity, many humans still look at the code at some point.
Re: Did Claude increase bugs in rsync?
#190Earlier quoted context omitted.
The em dashes are fine. If someone gives them shit about their writing, that's on the critic for being shitty. If they use AI to write, that's on them for being fake. But, to write online at all requires being ready to have people be shitty to you and ideally not reacting in a way that makes the situation worse. Sounds like they need work on that part. Anyway it is basically always possible for someone to find someth…
I don't think that you can successfully dismiss an obvious AI writing marker with "No these are fine, now look over there!! " Pay no attention to the man behind the curtain?
What? You are confused--human beings write em dashes also. Also you're being a dick to the OP, grow up.