Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

141–150 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#141

I'd be really interested to run AI detectors on essays from years before the ChatGPT era, just to see if anything gets flagged.

Yes, 3 out of 500 essays were flagged as 100% AI generated. There is a paragraph in the linked article about it.

This study is not very good frankly. Before ChatGPT there was Davinci and other model families which ChatGPT (what became GPT 3.5) was ultimately based on and they are the predecessors of today's most capable models. They should test it on work that is at least 10 to 15 years old to avoid this problem.

Re: Do AI detectors work? Students face false cheating accusations

#142

Earlier quoted context omitted.

How are you verifying you're correct? How do you know you're not finding false positives?

Have you tried reading AI-generated code? Most of the time it's painfully obvious, so long as the snippet isn't short and trivial.

To me it is not obvious. I work with junior level devs and have seen a lot of non-AI junior level code.

Re: Do AI detectors work? Students face false cheating accusations

#143
AI detectors do not work. I have spoken with many people who think that the particular writing style of commercial LLMs (ChatGPT, Gemini, Claude) is the result of some intrinsic characteristic of LLMs - either the data or the architecture. The belief is that this particular tone of 'voice' (chirpy sycophant), textual structure (bullet lists and verbosity), and vocab ('delve', et al) serves and and will continue to serve as an easy identifier of generated content.

Unfortunately, this is not the case. You can detect only the most obvious cases of the output from these tools. The distinctive presentation of these tools is a very intentional design choice - partly by the construction of the RLHF process, partly through the incentives given to and selection of human feedback agents, and in the case of Claude, partly through direct steering through SA (sparse autoencoder activation manipulation). This is done for mostly obvious reasons: it's inoffensive, 'seems' to be truth-y and informative (qualities selected for in the RLHF process), and doesn't ask much of the user. The models are also steered to avoid having a clear 'point of view', agenda, point-to-make, and on on, characteristics which tend to identify a human writer. They are steered away from highly persuasive behaviour, although there is evidence that they are extremely effective at writing this way (https://www.anthropic.com/news/measuring-model-persuasivenes...). The same arguments apply to spelling and grammar errors, and so on. These are design choices for public facing, commercial products with no particular audience.

An AI detector may be able to identify that a text has some of these properties in cases where they are exceptionally obvious, but fails in the general case. Worse still, students will begin to naturally write like these tools because they are continually exposed to text produced by them!

You can easily get an LLM to produce text in a variety of styles, some which are dissimilar to normal human writing entirely, such as unique ones which are the amalgamation of many different and discordant styles. You can get the models to produce highly coherent text which is indistinguishable from that of any individual person with any particular agenda and tone of voice that you want. You can get the models to produce text with varying cadence, with incredible cleverness of diction and structure, with intermittent errors and backtracking and _anything else you can imagine. It's not super easy to get the commercial products to do this, but trivial to get an open source model to behave this way. So you can guarantee that there are a million open source solutions for students and working professionals that will pop up to produce 'undetectable' AI output. This battle is lost, and there is no closing pandora's box. My earlier point about students slowly adopting the style of the commercial LLMs really frightens me in particular, because it is a shallow, pointless way of writing which demands little to no interaction with the text, tends to be devoid of questions or rhetorical devices, and in my opinion, makes us worse at thinking.

We need to search for new solutions and new approaches for education.

Re: Do AI detectors work? Students face false cheating accusations

#144

We had a time when CGI took off, where everything was too polished and shiny and everyone found it uncanny. That started a whole industry to produce virtual wear, tear, dust, grit and dirt. I wager we will soon see the same for text. Automatic insertion of the right amount of believable mistakes will become a thing.

You can already do that easily with ChatGPT. Just tell it to rate the text it generated on a scale from 0-10 in authenticity. Then tell it to crank out similar text at a higher authenticity scale. Try it.

Re: Do AI detectors work? Students face false cheating accusations

#145

Earlier quoted context omitted.

Do you think it stupid to scan kids for weapons, or stupid to think that a metal detector will find weapons?

I think it's stupid to have a country where guns are legal.

Guns are legal in almost every country - I think your problem is with countries that have almost no restriction on gun ownership. e.g. Here in the UK you can legally own a properly licensed rifle or shotgun and even a handgun in some places outside of Great Britain (e.g. Northern Ireland).

Re: Do AI detectors work? Students face false cheating accusations

#146

For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy. Text seems to use the same general framework (although words are swapped around) also we see what I call 'word of the week' where whichever 'AI' engine seems to get hung up on a particular English word which is often an unusual one and uses it at every opportunity. It isn't long before you real…

the students are too lazy and dumb to do their own thinking and resort to ai. the teachers are also too lazy and dumb to assess the students' work and resort to ai. ain't it funny?

Re: Do AI detectors work? Students face false cheating accusations

#147

Earlier quoted context omitted.

I went to public schools in middle class neighborhoods in California from the late sixties to the early eighties. My teachers were largely excellent. I think that was due to cultural and economic factors - teaching was considered a profession for idealistic folks to go into at the time and the spread between rich and poor was less dramatic in the 50s and 60s (when my teachers were deciding their professions). So the…

It was the tail end of when smart women had few intellectually stimulating options and teacher was a decent choice.

[flagged]

Re: Do AI detectors work? Students face false cheating accusations

#148
post #138
post #129

Earlier quoted context omitted.

> For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy. Text seems to use the same general framework (although words are swapped around) also we see what I call 'word of the week' Easy to catch people that aren't trying in the slightest not to get caught, right? I could instead feed a corpus of my own writing to ChatGPT and ask it to write in my s…

I don't believe it's possible at all if any effort is made beyond prompting chat-like interfaces to "generate X". Given a hand crafted corpus of text even current llms could produce perfect style transfer for a generated continuation. If someone believes it's trivially easy to detect, then they absolutely have no idea what they are dealing with. I assume most people would make least amount of effort and simply prompt…

Are you then plagiarising if the LLM is just regurgitating stuff you’d personally written?

The point of these detectors is to spot stuff the students didn’t research and write themselves. But if the corpus is your own written material then you’ve already done the work yourself.

Re: Do AI detectors work? Students face false cheating accusations

#149
post #138

Earlier quoted context omitted.

I don't believe it's possible at all if any effort is made beyond prompting chat-like interfaces to "generate X". Given a hand crafted corpus of text even current llms could produce perfect style transfer for a generated continuation. If someone believes it's trivially easy to detect, then they absolutely have no idea what they are dealing with. I assume most people would make least amount of effort and simply prompt…

Are you then plagiarising if the LLM is just regurgitating stuff you’d personally written? The point of these detectors is to spot stuff the students didn’t research and write themselves. But if the corpus is your own written material then you’ve already done the work yourself.

LLM is just regurgitating stuff as a principle. You can request someone else's style. People who are easy to detect simply don't do that. But they will learn quickly

Re: Do AI detectors work? Students face false cheating accusations

#150

Earlier quoted context omitted.

I think it's stupid to have a country where guns are legal.

Guns are legal in almost every country - I think your problem is with countries that have almost no restriction on gun ownership. e.g. Here in the UK you can legally own a properly licensed rifle or shotgun and even a handgun in some places outside of Great Britain (e.g. Northern Ireland).

Just because something is technically legal, doesn't mean it's in any way common or part of UK culture to own a gun.

There hasn't been a school shooting in the UK for nearly 30 years. Handguns were banned after the last school shooting and there hasn't been one since.

https://en.wikipedia.org/wiki/Category:School_shootings_in_t...

Although that fact is sometimes forgotten by schools who copy the US in having "active shooter drills" though. Modern schools sound utterly miserable.

Post reply on HN