Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Did Claude increase bugs in rsync?

alexispurslane.github.io

Post reply on HN
Print view View original

41–50 of 611 posts

Re: Did Claude increase bugs in rsync?

#41

Post by gadrev »

logicprog wrote: post #34
gadrev wrote: post #29

Ok. $ apt-cache policy rsync | grep Installed Installed: 3.4.1+ds1-7ubuntu0.2 $ sudo apt-mark hold rsync rsync set on hold.

Did you face any actual bugs or regressions? Or are you doing this just because of the bandwagon that's going around right now? Because until you can actually present an argument for why this release is worse than any of the others, which is precisely the subject of my post, then this is not an argument against my post at all. This is just a self-referential appeal to authority.

Nah, I skimmed TFA but then I went into the linked GH issues thread, and that's the one that scared me a bit. I just want to hold it for a while and not run into some of the things I'm reading since I'm on the latest ubuntu. Just a precaution.

I didn't have the time to actually think about any "arguments" at all tbh it's just a knee jerk reaction as I get ready to log off for the weekend. Not actually looking to argument for or against your post at all lol.

Re: Did Claude increase bugs in rsync?

#43

Post by logicprog »

CuriouslyC wrote: post #28
logicprog wrote: post #10

Earlier quoted context omitted.

Some notes on this: - I used GLM 5.1 to help with the coding and math for this. - However, I explicitly dictated where the data should be pulled from (GitHub, Bugzilla, mailing list), how it should be tagged and grouped, and what data to look at (e.g. bugs instead of regressions) - Additionally, I consulted with my wife, who has a master's degree in statistics from Penn State University for what sort of statistical m…

I'd suggest writing the lead-in yourself and boxing AI prose separately from your prose in the analysis for future articles. You can give the humanized summary/eli5/key points, then have "details according to AI" boxes that go into nitty-gritty. People seem to dislike AI ghostwriting, but most of these people still use AI, so perhaps keeping authorship clear and separate will avoid some of the flak.

This seems fair. Of course, now that I've posted this here once, I doubt it'll get constructive engagement again, but I can at least improve this for the future

Re: Did Claude increase bugs in rsync?

#44

Post by sanitycheck »

logicprog wrote: post #35
sanitycheck wrote: post #31

Earlier quoted context omitted.

I think the point people are making is that when the text has an "AI smell" (it does), we immediately lose trust in the veracity of any claim being made and feel like continuing to read what is possibly a hallucinated fiction is a complete waste of time. At this point we're all used to skimming through thousands of AI-generated sentences every working day and constantly thinking "this is likely to be 20% bullshit", i…

Do you think it would help if I went through and manually rewrote all of the prose? If it would get people to listen, I'd be totally willing to do it. It's not like I don't like writing. I just was focused on something else when I was making this, namely trying to find a good methodology that isn't insane for this low amount of data.

I'm pretty sure more people would read it to the end if it didn't seem like AI output, yes.. At the very least you would have fewer (maybe not 0!) comments here saying it's AI slop.

Re: Did Claude increase bugs in rsync?

#45

Post by tappio »

dang wrote: post #91

[stub for offtopicness] [see https://news.ycombinator.com/item?id=48416020 for how all this happened in the first place]

A lot of people criticizing because it's heavily written with LLM, but I mean, if someone produced this piece pre-LLM, would they criticize it? is the critique due to use of LLM or due to the content being truly hard to follow? I read it and I would say, there are some problems with the writing, but its not a bad piece.

Of course this is a bigger problem, as its now harder to distinguish content that is "AI slop" with "content co-authored with AI that is carefully reviewed" with a quick glimpse, and the "AI smell" is quite off-putting. My initial reaction was also negative, but after glimpsing it through and reading the summaries, I found it decent summary, which also... speaks of this thread, of the content of the blog post and everything about the discussion and the strong feelings people have developed around the use of LLMs.

Anyhow, it would be good to disclose the repo with the code for the statistics & use of LLM in the writing right up front. Which model, and why it was used to do the writing, etc. Its enough to say "I think it writes better than I do" or "I was in a hurry, sorry" or what ever, but it really should be disclosed. It reads more honest.

ps. really... that sideways scroll? plz fix it.

Re: Did Claude increase bugs in rsync?

#46

Post by logicprog »

bradrn wrote: post #40
logicprog wrote: post #35

Earlier quoted context omitted.

Do you think it would help if I went through and manually rewrote all of the prose? If it would get people to listen, I'd be totally willing to do it. It's not like I don't like writing. I just was focused on something else when I was making this, namely trying to find a good methodology that isn't insane for this low amount of data.

Yes, that would help considerably. (Also, I suggest clearly acknowledging where AI was/wasn’t used. I like CuriosityC’s suggestion: https://news.ycombinator.com/item?id=48411968 )

Alright, I'll do that. Although, sadly, I already posted it here, so I won't be able to post it again — I'll be stuck with this trash comments section that doesn't deal with any of the actual claims, just the aesthetics.

Re: Did Claude increase bugs in rsync?

#47

Post by JasonSage »

logicprog wrote: post #35
sanitycheck wrote: post #31

Earlier quoted context omitted.

I think the point people are making is that when the text has an "AI smell" (it does), we immediately lose trust in the veracity of any claim being made and feel like continuing to read what is possibly a hallucinated fiction is a complete waste of time. At this point we're all used to skimming through thousands of AI-generated sentences every working day and constantly thinking "this is likely to be 20% bullshit", i…

Do you think it would help if I went through and manually rewrote all of the prose? If it would get people to listen, I'd be totally willing to do it. It's not like I don't like writing. I just was focused on something else when I was making this, namely trying to find a good methodology that isn't insane for this low amount of data.

When there's no discernable human filter on the text output, reading the text suggests it's what the LLM produced and not what a human considered.

This is low-quality--every single day I witness Codex and Claude misunderstand, mislead, and hallucinate responses based on "assumptions" and I have to fact-check them.

If I wanted a statistical analysis and to be the human in the loop, I would ask the LLM myself, and I would definitely NOT read an article that just dumps the LLM output as-is.

Re: Did Claude increase bugs in rsync?

#48

Post by logicprog »

tappio wrote: post #45
dang wrote: post #91

[stub for offtopicness] [see https://news.ycombinator.com/item?id=48416020 for how all this happened in the first place]

A lot of people criticizing because it's heavily written with LLM, but I mean, if someone produced this piece pre-LLM, would they criticize it? is the critique due to use of LLM or due to the content being truly hard to follow? I read it and I would say, there are some problems with the writing, but its not a bad piece. Of course this is a bigger problem, as its now harder to distinguish content that is "AI slop" wit…

Thank you for your constructive input, you're one of only a few others here who had any. I'll definitely do that. I didn't think, since the output was templated directly from the numbers generated by a reproducible python script, that people would get so up in arms about the aesthetics, but I guess I forgot to say that.

Re: Did Claude increase bugs in rsync?

#49

Post by BigTTYGothGF »

logicprog wrote: post #18
mschuster91 wrote: post #8

Earlier quoted context omitted.

This article reeks of LLM "assistance" at the very least. Please, why can't people write stuff by hand themselves any more? It's a good analysis but how can I trust it without reviewing everything myself?!

I mean, you can literally clone my repo, run the Python that rebuilds the database and does the whole data analysis and to end from scratch, and verify that the numbers are accurate. I made the code for this analysis public for that exact reason. This wasn't just an LLM running unsupervised in a loop. I came up with the methodologies and metrics and data scraping strategies precisely myself, iterated on it to try to…

> I mean, you can literally

You didn't care enough to make a good writeup, why should we believe that you cared enough to make a good analysis?

Re: Did Claude increase bugs in rsync?

#50

Post by aozgaa »

jchw wrote: post #15
logicprog wrote: post #10

Earlier quoted context omitted.

Some notes on this: - I used GLM 5.1 to help with the coding and math for this. - However, I explicitly dictated where the data should be pulled from (GitHub, Bugzilla, mailing list), how it should be tagged and grouped, and what data to look at (e.g. bugs instead of regressions) - Additionally, I consulted with my wife, who has a master's degree in statistics from Penn State University for what sort of statistical m…

I really struggle to believe you wrote text like: > A simple distributional analysis of every rsync release with bug data. No model. No assumptions. Just placement.

In general, it seems HN does not like to read llm-generated articles. I ran into this myself when using an llm to edit some stuff I wrote.

At the time, I found this a bit irritating, but with a few weeks time I see the merit. The informational content tends to fall into “derivative” territory when LLM’s write stuff. And people are here for novelty and some socialization.

Also LLM prose seems optimized for engagement rather than concise communication. Takes longer to sift through linguistic boilerplate to get to the point. (The quoted bit being a case in point)

Post reply on HN