Of course they don't work. The whole point of LLMs is that they're indistinguishable from human-written text. People need to understand that AI isn't magic, it can't tell that something wasn't written by a human without a distinctive clue.
Do AI detectors work? Students face false cheating accusations
381–390 of 1001 posts
Re: Do AI detectors work? Students face false cheating accusations
#382Earlier quoted context omitted.
An essay written under examination conditions is fine. We don’t need new assessment techniques. We have known how to asses that a student and that student alone for centuries.
My ability to write an essay under exam conditions is...poor. Thankfully there were less than a handful of essays I had to write as part of my undergraduate CS degree and I only remember one under exam conditions. I think it's probably more concerning that spitting out the most generic mathematically formulaic bullshit on a subject is likely to get a decent mark. In that case what are we actually testing for?
Conformance.
Re: Do AI detectors work? Students face false cheating accusations
#383Earlier quoted context omitted.
let us ban knives then... got a license for that mate?
There's already a UK ban on carrying knives in public unless you have an occupational need and they're wrapped up or at least not just sitting in your pocket. Licensing wouldn't be worthwhile as almost every household would want knives for food preparation.
[deleted]
Re: Do AI detectors work? Students face false cheating accusations
#384FWIW, I'm a consultant for a large University hospital, and Dutch. My PhD thesis, years ago, got the remark: "Should have checked with a native speaker." So, now I use ChatGPT to check my English. I just write what I want to write than ask it to make my text more "More concise, business-like and not so American" (yeah the thing is by default as ultra enthusiastic as an American waiter). And 9 out of 10 times it says…
I completely agree. LLMs are incredibly useful for improving the flow and structure of an argument, not just for non-native speakers, but even for native English speakers.
Making texts more accessible through clear language and well-structured arguments is a valuable service to the reader, and I applaud anyone who leverages LLMs to achieve that. I do the same myself.
Re: Do AI detectors work? Students face false cheating accusations
#385My daughter’s 7th grade work is 80% flagged as AI. She is a very good writer, it’s interesting to see how poorly this will go. Obviously we will go back to in class writing.
The article demonstrates that good, simple prose is being flagged as AI-generated. Reminds me of a misguided junior high English teacher that half-heartedly claimed I was a plagiarist for including the word "masterfully" in an essay, when she knew I was too stupid to use a word like that. These tools are industrializing that attitude and rolling it to teachers that otherwise wouldn't feel that way.
> she knew I was too stupid to use a word like that.
Oh... It is the story of my school math education. I always got bad marks, because I was "too stupid to come up with this particular solution to the problem". I didn't thought it was really unfair, because I thought myself to be lazy, and I looked for such solutions to math problems that would minimize my work. Oftentimes I ignored textbook ways to solve problems and used my own. I believed that it was a cheating, so naturally I got worse marks, but I put up with that, because I was lazy to do it in more complex way from a textbook.
Re: Do AI detectors work? Students face false cheating accusations
#386It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…
Written assay evaluation is not and has never been an effective evaluation. It was always a cost saving measure because allocating 30min face to face time with each individual student for each class is such a gigantic cost for the institution that they cannot even imagine doing it. Think about that the next time you look at your student debt, it couldn’t even buy you 30min time per class individually with the teacher…
You can still do written essay evaluations. You could just require proctored exams whether or not you use software like Examsoft. If it's a topic that benefits from writing from a store of material, you can permit students to bring either unlimited supplemental printed material or a limited body of printed material into the exam room.
For longer essays, you can just build in an oral examination component. This face time requirement is just not that hard to include given that even in lecture hall style settings you can rely on graduate student TAs who do not really cost anything. The thing is that the universities don't want to change how they run things. Adjuncts in most subjects don't cost anything and graduate students don't cost anything. They earn less than e.g. backroom stocking workers. This is also why they, by and large, all perform so poorly. 30 minutes of examiner time costs maybe $11 or less. Even for a lecture class with 130 students, that's under $1,500. Big woop.
There are some small changes to grading practices that would make life very hard for AI cheaters, such as even cite checking a portion of citations in an essay. The real problem is that US universities are Soviet-style institutions in which gargantuan amounts of cash are dumped upon them and they pretend to work for it while paying the actual instructors nothing.
Re: Do AI detectors work? Students face false cheating accusations
#387Well, that's ONE problem. "The Elite College Students Who Can’t Read Books" https://www.theatlantic.com/magazine/archive/2024/11/the-eli...
Books? You mean neoluddist obsolete devices?? We have audiobooks now cmon !!!!!!
Re: Do AI detectors work? Students face false cheating accusations
#388Earlier quoted context omitted.
Oh yes. When I'm teaching a class of 200 students it's totally plausible that we're going to do 10 15 minute one on one conversations with every student. Because that's only 20 days non stop with no sleep. We would need to increase the amount of teaching staff by well over 10x to do this. The costs would be astronomical.
>The costs would be astronomical. Countries have no problem spending astronomical amounts on old people. If the country wants productive young people, the country will find a way.
We’ve already found a way: it’s called “mass immigration.”
Why bother training and educating the young people who are already here when you can just import them from poorer countries?
Re: Do AI detectors work? Students face false cheating accusations
#389For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy. Text seems to use the same general framework (although words are swapped around) also we see what I call 'word of the week' where whichever 'AI' engine seems to get hung up on a particular English word which is often an unusual one and uses it at every opportunity. It isn't long before you real…
> also we see what I call 'word of the week' where whichever 'AI' engine seems to get hung up on a particular English word which is often an unusual one and uses it at every opportunity So do humans. Many people have pet phrases or words that they use unusually often compared to others.
People have their favorite phrases or words, but also as readers we fixate on words that we don't personally use, and project that onto the writer.
But as a second language learner, you notice that people get stuck on particular words during writing sessions. If I run into a very unusual (and unnecessary) word, I know they're going to use it again within a page or two, maybe once after that, then never again.
I blame it on the writer remembering a cool word, or finding a cool word in a thesaurus, then that word dropping out of their active vocabulary after they tried it out a couple times. There's probably an analogue in LLMs, if just because that makes unusual words more likely to repeat themselves in a particular passage.
Re: Do AI detectors work? Students face false cheating accusations
#390It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…
>AI is here to stay; new methods should be used to assess student performance.
Here is the brand new method - asking verbal questions in person and evaluating answers. Also allow high tech aides in the form of chalk and blackboard