Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we measured AI writing across arXiv, and where the measurement breaks

unslop.run

Post reply on HN
Print view View original

181–185 of 185 posts

Re: How we measured AI writing across arXiv, and where the measurement breaks

#181

Post by JadeNB »

jazzyjackson wrote: post #168
JadeNB wrote: post #59

Earlier quoted context omitted.

Which part is a sin? Using LLMs to deal with a lack of English-language fluency? I am a scientist (actually a mathematician, if it matters), and, if that's the way to deal with the practical hegemony of English in the scientific literature, then I have no problem with it. Rather that than people with important ideas can't get them before the scientific community. As long as the authors personally check and stand behi…

It would be nice if scientists could write and publish in their first language. If a machine translation brings it to a wider audience fine, but the lack of disclosure of who authored the paper is a problem.

I'd way rather read a translation in which the author themselves has played a part, and at least nominally signed off on it, than one done on the fly with even less than usual guarantee of its correctness.

(I agree that authors should be able to publish in their first language, but I also note that the historical practicality has been that publication in non-dominant languages has resulted more in silos than in broadening access.)

Re: How we measured AI writing across arXiv, and where the measurement breaks

#182

Post by grey-area »

chungusamongus wrote: post #174
grey-area wrote: post #100

Earlier quoted context omitted.

It certainly casts suspicion on the ‘findings’ which may well be hallucinated or subtly wrong.

Because humans totally don't get things wrong or fabricate lies. Even the term "hallucination" is a clumsy way to describe what is happening when an LLM spits out an unsound statement. What you are doing is more akin to a hallucination than what the LLM is doing. Even in the process of criticizing this technology, you cannot help but to anthropomorphize it.

* Because humans totally don't get things wrong or fabricate lies.*

Not in the same way no. Why so trenchant in defending a word generator?

If you prefer for hallucinate substitute ‘generate plausible but incorrect text’. I don’t think of LLMs as anything approaching human, sorry to disappoint.

Re: How we measured AI writing across arXiv, and where the measurement breaks

#183

Post by jsrozner »

jrm4 wrote: post #170
jsrozner wrote: post #36

Earlier quoted context omitted.

If as a primary school student you had needed to reverify every fact /experiment that was presented in your textbooks, you would not have gotten very far. Trust is very important to human progress.

Absolutely. Which is why what I think I'm saying is that -- it's not ncessarily important who or what WROTE the paper, it's simply important that human-in-the-loop is strong.

Right, but even careful reading cannot validate the experimental setup.

Re: How we measured AI writing across arXiv, and where the measurement breaks

#184

Post by chungusamongus »

grey-area wrote: post #182
chungusamongus wrote: post #174

Earlier quoted context omitted.

Because humans totally don't get things wrong or fabricate lies. Even the term "hallucination" is a clumsy way to describe what is happening when an LLM spits out an unsound statement. What you are doing is more akin to a hallucination than what the LLM is doing. Even in the process of criticizing this technology, you cannot help but to anthropomorphize it.

* Because humans totally don't get things wrong or fabricate lies.* Not in the same way no. Why so trenchant in defending a word generator? If you prefer for hallucinate substitute ‘generate plausible but incorrect text’. I don’t think of LLMs as anything approaching human, sorry to disappoint.

I'm not "defending a word generator." The tool does not have agency. I'm saying that just because the author of a study uses an LLM, that doesn't mean the study is incorrect. Even in your weird attempts to form a retort, you are still unwittingly assigning agency to a tool. If a tool is used incorrectly, if the author does not check their own work, that's on them. And do you think an author who makes such mistakes would be more competent without an LLM at their disposal? Please. People make careless mistakes, they fabricate data, etc.

>If you prefer for hallucinate substitute ‘generate plausible but incorrect text’.

Yes I prefer accurate statements as opposed to anthropomorphizing a tool, which is what you were doing before.

Post reply on HN