Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Did Claude increase bugs in rsync?

alexispurslane.github.io

Post reply on HN
Print view View original

21–30 of 611 posts

Re: Did Claude increase bugs in rsync?

#21

Post by rroblak »

roywiggins wrote: post #7
dang wrote: post #91

[stub for offtopicness] [see https://news.ycombinator.com/item?id=48416020 for how all this happened in the first place]

> A simple distributional analysis of every rsync release with bug data. No model. No assumptions. Just placement. If you want me to read your analysis, you are going to have to make it not read like Claude wrote it. What does "placement" even mean here?

Yeah, made me chuckle that an LLM— probably Claude— was used to write this.

The use of "regime shift" is what gave it away for me. I've never seen a human write that, but Claude does from time to time.

At least they removed occurrences of "load-bearing".

Re: Did Claude increase bugs in rsync?

#22

Post by faitswulff »

> The analysis uses a single metric: bugs per 10 commits (bugs/10c).

Bugs per commit as a metric papers over severity, both in terms of security severity as well as the effect on the user. A mislabeled button has the same weight as the entire app crashing in this framework.

Re: Did Claude increase bugs in rsync?

#23

Post by logicprog »

jchw wrote: post #15
logicprog wrote: post #10

Earlier quoted context omitted.

Some notes on this: - I used GLM 5.1 to help with the coding and math for this. - However, I explicitly dictated where the data should be pulled from (GitHub, Bugzilla, mailing list), how it should be tagged and grouped, and what data to look at (e.g. bugs instead of regressions) - Additionally, I consulted with my wife, who has a master's degree in statistics from Penn State University for what sort of statistical m…

I really struggle to believe you wrote text like: > A simple distributional analysis of every rsync release with bug data. No model. No assumptions. Just placement.

No, I didn't write the text itself. I'm typically significantly more verbose and elliptical, and more than that, the numbers and methodology changed often enough over the course of the last couple days I was working on this because I was trying to get it to be as accurate and fair as possible that trying to keep the whole thing up to date manually would have been problematic.

Re: Did Claude increase bugs in rsync?

#25

Post by gamegod »

roywiggins wrote: post #7
dang wrote: post #91

[stub for offtopicness] [see https://news.ycombinator.com/item?id=48416020 for how all this happened in the first place]

> A simple distributional analysis of every rsync release with bug data. No model. No assumptions. Just placement. If you want me to read your analysis, you are going to have to make it not read like Claude wrote it. What does "placement" even mean here?

It's the ultimate product for marketers. It inserts itself as an advertisement into every conversation now and defends itself against criticism. Just crazy. There's no hope for the rest of us.

Re: Did Claude increase bugs in rsync?

#26

Post by [deleted] »

jchw wrote: post #15
logicprog wrote: post #10

Earlier quoted context omitted.

Some notes on this: - I used GLM 5.1 to help with the coding and math for this. - However, I explicitly dictated where the data should be pulled from (GitHub, Bugzilla, mailing list), how it should be tagged and grouped, and what data to look at (e.g. bugs instead of regressions) - Additionally, I consulted with my wife, who has a master's degree in statistics from Penn State University for what sort of statistical m…

I really struggle to believe you wrote text like: > A simple distributional analysis of every rsync release with bug data. No model. No assumptions. Just placement.

[deleted]

Re: Did Claude increase bugs in rsync?

#27

Post by noctuid »

jchw wrote: post #15
logicprog wrote: post #10

Earlier quoted context omitted.

Some notes on this: - I used GLM 5.1 to help with the coding and math for this. - However, I explicitly dictated where the data should be pulled from (GitHub, Bugzilla, mailing list), how it should be tagged and grouped, and what data to look at (e.g. bugs instead of regressions) - Additionally, I consulted with my wife, who has a master's degree in statistics from Penn State University for what sort of statistical m…

I really struggle to believe you wrote text like: > A simple distributional analysis of every rsync release with bug data. No model. No assumptions. Just placement.

[flagged]

Re: Did Claude increase bugs in rsync?

#28

Post by CuriouslyC »

logicprog wrote: post #10
dang wrote: post #91

[stub for offtopicness] [see https://news.ycombinator.com/item?id=48416020 for how all this happened in the first place]

Some notes on this: - I used GLM 5.1 to help with the coding and math for this. - However, I explicitly dictated where the data should be pulled from (GitHub, Bugzilla, mailing list), how it should be tagged and grouped, and what data to look at (e.g. bugs instead of regressions) - Additionally, I consulted with my wife, who has a master's degree in statistics from Penn State University for what sort of statistical m…

I'd suggest writing the lead-in yourself and boxing AI prose separately from your prose in the analysis for future articles. You can give the humanized summary/eli5/key points, then have "details according to AI" boxes that go into nitty-gritty. People seem to dislike AI ghostwriting, but most of these people still use AI, so perhaps keeping authorship clear and separate will avoid some of the flak.

Re: Did Claude increase bugs in rsync?

#30

Post by logicprog »

gamegod wrote: post #25
roywiggins wrote: post #7

Earlier quoted context omitted.

> A simple distributional analysis of every rsync release with bug data. No model. No assumptions. Just placement. If you want me to read your analysis, you are going to have to make it not read like Claude wrote it. What does "placement" even mean here?

It's the ultimate product for marketers. It inserts itself as an advertisement into every conversation now and defends itself against criticism. Just crazy. There's no hope for the rest of us.

It's not defending itself here, both because I used GLM 5.1, not Claude, and because I was the one who decided to do this analysis, iterated through six or seven different methodologies to try to find the one that was most honest with the data that I had (all of the methodologies showed directionally and often in magnitude the exact same thing, but I wanted to do something that fit the purpose, in consultation with my wife, who, as I've mentioned elsewhere, has a master's degree in statistics), and, of course, I specifically chose all of the metrics and sources for the data.

If you don't want to read the LLM prose, you can just go to the GitHub of my project, grab the scripts, and run the full pipeline. It will gather the data, build the database, and run the analysis from scratch for you, and you can look at the numbers directly. It's all repeatable.

Post reply on HN