Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we measured AI writing across arXiv, and where the measurement breaks

unslop.run

Post reply on HN
Print view View original

131–140 of 185 posts

Re: How we measured AI writing across arXiv, and where the measurement breaks

#131

Post by addaon »

William_BB wrote: post #123
addaon wrote: post #106

Earlier quoted context omitted.

> It's hard to say if this code is structurally better or worse than before, but it's certainly voluminous and as far as leadership could ever tell, with their flawed metrics, that is all that matters. Hah. I've been working with ChatGPT-5.6 Sol and a lot of guidance to port a particular geometry from 2800 lines of Python-scripted Rhino3D (basically a custom parametric CAD kernel in there, plus use of it) to Python-s…

> I think "absolute crap but still useful to me" is a pretty high value and not worth neglecting If I extrapolate this example to my professional life, this code now manages millions of dollars, a single mistake can wipe it all out, it has to be maintained by 5 other engineers and understood by 5 other domain experts.

That's a real risk, yes. There's definitely a difference between cases where the code is the artifact being built, and cases where the code is being built to generate an artifact that is independently valuable and assessable. I'm happily in the latter case -- but I definitely share your fear of those who think current LLMs are valuable in the former case.

Re: How we measured AI writing across arXiv, and where the measurement breaks

#132

Post by brokenodo »

leawi wrote: post #130

Is it just me, or does this article itself read as AI generated?.. Ironically, I tried running the text of this article through their own classifier, and pretty much all of it was highlighted in red. I'm not sure what to make of this. Is this some kind of intentional irony?

It obviously is! Human pattern-matching abilities are rather amazing, and unfortunately I've seen so much AI text that it instantly flagged in my brain within 3 seconds of opening the page.

Pangram agrees: https://www.pangram.com/history/3de33376-94e3-404d-bbb0-751a...

Re: How we measured AI writing across arXiv, and where the measurement breaks

#133

Post by aionwikipedia »

willquack wrote: post #17

> If a tool marks 40% of new papers as machine-written but also marks 20% of papers written before ChatGPT existed, the real story is the 20% nobody mentioned. When 65% of the papers you read have the characteristics of being AI written, whether or not you use AI to write, your writing will be influenced by the AI style. I imagine this must be particularly the case for newbie researchers who are still developing thei…

even if your writing is influenced by "the AI style," it would likely only show up in patches, a word or rhetorical flourish maybe, but AI-generated text doesn't "sound like AI" in patches, it does so consistently across the entirety of the output. it's very hard for a person to maintain the syntactical patterns of AI writing over an extended period of time unless they know exactly what those are -- probably more knowledge than any person currently has, really -- and are applying a level of detail to each clause comparable to forging a painting.

there's also the fact that "the AI style" has changed over time. for instance, the word "delve" is notorious as an "AI sign," which it was up until mid-2024, at which point it dropped off sharply and has now basically disappeared from LLM output. so if someone happened to pick that up due to reading it everywhere, their writing is now less characteristic of AI, not more.

Re: How we measured AI writing across arXiv, and where the measurement breaks

#134

Post by simonreiff »

Paracompact wrote: post #94
dopamine_daddy wrote: post #8

Earlier quoted context omitted.

I tried my best to avoid leakage. If you're curious about how I trained the detector I have a writeup on it: https://unslop.run/blog/how-our-ai-text-detector-works FYI this is all relatively new so there might be lots of issues and iterations coming.

My honest first impressions, since I think the project is well intended: This writeup is itself AI, and I would venture to call it slop. The Calibration section is borderline uninterpretable, and I challenge any non-author who claims to understand it to answer some basic peer review questions about it.

I have to concur, unfortunately. This isn't reliable, reproducible, or interpretable.

Re: How we measured AI writing across arXiv, and where the measurement breaks

#135

Post by edot »

leawi wrote: post #130

Is it just me, or does this article itself read as AI generated?.. Ironically, I tried running the text of this article through their own classifier, and pretty much all of it was highlighted in red. I'm not sure what to make of this. Is this some kind of intentional irony?

Yes, 100%. "where the measurement breaks" and "honest account of the limitations" are all I needed to see.

Re: How we measured AI writing across arXiv, and where the measurement breaks

#136

Post by wxw »

Not convinced that this slop measurement is useful.

I asked Codex to generate an article with a high score and then asked Codex to (reverse?) hill climb that score. The original generation scored 97% and then the optimized one scored 1%. Both are pretty bad and read like slop.

https://gist.github.com/wbew/8a2bd6686bf875210f2244ac8ea65bf...

Re: How we measured AI writing across arXiv, and where the measurement breaks

#138

Post by pcrh »

ryandvm wrote: post #85

There some real game theory mechanics at play when it comes to LLM usage in corporations. Devs are cranking out superficially superior code and documentation by just aiming the Claude Code fire hose at everything they can. Leadership encourages this because from what they can tell, there is no downside. It's hard to say if this code is structurally better or worse than before, but it's certainly voluminous and as far…

>I can't imagine that the same factors aren't at play in the scientific research community

I can't speak for the entire scientific research community, but I can say that for basic biomedical research (not clinical research), LLMs are mostly ignored. They simply don't have the ability to parse most raw wet lab data.

This obviously has not stopped the creation of a legion of startups, new Asst Profs, etc, claiming that they are using "AI" to crack new frontiers. In practice, the most effective of these use machine learning, rather than LLMs.

Re: How we measured AI writing across arXiv, and where the measurement breaks

#139

Post by simonreiff »

zfnmxt wrote: post #56
lingeringpine wrote: post #43

I am not a native English speaker. This is not surprising to me. I think most of the papers we write would be flagged by AI detectors. It is not because we ask LLM to write us a paper about X. It is because we are bad at writing in a scientific style, and american editors expect us to do it. With LLMs, we can write in basic sentences and tell the LLM the idea and it converts that to nice writing. If you write each pa…

> If you write each paragraph and have an LLM make that paragraph more scientific, it is entirely your paper "More scientific" is not some merely stylistic thing that faithfully preserves the original meaning of what you wrote. The precise details of each paragraph matters a lot in terms of what and how it communicates. The fact that these details do matter means that, according to my accounting, it is not entirely y…

Being charitable, "more scientific" means more rigorous, reproducible, reliable based on the consensus of that scientific community (if any consensus exists, which it usually does in math and hard sciences), transparent in methodology, and candid about authorities cited and limitations to the experimental approach. Science is not a style guide; the purpose of a research publication is to impart a result to experts in the field and make claims that can be rigorously scrutinized. The best way to make a claim credibly is to demonstrate full fluency with the tools and techniques of the academic community in applying them thoughtfully to the chosen problem. But the result of doing that often looks downright impenetrable to the general public; see, e.g., Niven 1947 (1-page proof that pi is irrational). It's not the point of science to make it accessible to everyone. The point is to convince experts in the field of the correctness of the claim. The audiences are often totally different, and the tone and qualities of the writings likewise must vary.
Post reply on HN