Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

Post reply on HN
Print view View original

461–470 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#461

Post by jerf »

mikeyinternews wrote: post #458

One of my kid's teachers sent out a warning to students that all essays would be checked with AI detection software and the repercussions one would face if caught. A classmate did an AI check on the teacher's warning and it came back positive for having been AI-generated.

The default tone of ChatGPT and the default tone of school or academic writing (at all levels) are not exactly the same, but in the grand vector space of such things, they are awfully close to each other. And all the LLMs have presumably already been fed with an awful lot of this sort writing, too. It's not a surprise that a by-the-numbers report, either in high school or college, of the sort that generally ought to get a good grade because it is exactly what is being asked for, comes out with a high probability of having been generated by GPT-style technology. And I'm sure LLMs have been fed with a lot of syllabuses and other default teacher writing documents, and almost any short teacher-parent or teacher-student communication is not going to escape from same basin of writing attraction that the LLMs write in very easily.

Re: Do AI detectors work? Students face false cheating accusations

#462

Post by freilanzer »

raincole wrote: post #353
smartmic wrote: post #292

Earlier quoted context omitted.

I agree with you, but I would go further and turn the tables. An AI should simply not be allowed to evaluate people, in any context whatsoever. For the simple reason that it has been proven not to work (and will also never). Anyone interested to learn more about it, I recommend the recent book "AI Snake Oil" from Arvind Narayanan and Sayash Kapoor [1]. It is a critical but nuanced book and helps to see the whole AI h…

Statistical models (which "AI" is) have been used to evaluate people's outputs since forever. Examples: Spam detection, copyrighted material detection, etc.

But not in cheating or grades, etc. Spam filters are completely different from this.

Re: Do AI detectors work? Students face false cheating accusations

#463

Post by skydhash »

_fizz_buzz_ wrote: post #208
delusional wrote: post #199

Earlier quoted context omitted.

> I remember being told at school, that we weren't allowed to use calculators in exams I remember being told the same thing, but I happen to believe that it was a fantastic policy, with a lackluster explanation. The idea that you wouldn't have a calculator was obviously silly, even at the time, but underlying observation that relying on the calculator would rob you of the mental exercise the whole ordeal was supposed…

Very well put. I would actually suggest to not use calculators in high school anymore. They add very little value and if it is still the same as when I was in high school, it was a lot of remembering weird key combinations on a TI calculator. Simply make the arithmetic simple enough that a calculator isn't needed.

I don't remember exactly, but I think we were only allowed the simplest calculators in middle school (none before), and scientific calculators in high schools (mostly for the trigonometric and power functions). I got to use a TI in university, but never used it that much as I've got the basic function graphs memorized.

Re: Do AI detectors work? Students face false cheating accusations

#464

Post by naming_the_user »

lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

Nonsense.

You are in a room with a sheet of paper and a pen. Go.

You’re acting as if 2010 was a hundred years ago.

Re: Do AI detectors work? Students face false cheating accusations

#465

Post by kibwen »

junaru wrote: post #351
lm28469 wrote: post #339

Earlier quoted context omitted.

In France my essays were written in class, no phones, no book, just your brain, a sheet a paper and a pen. That's still 100% doable today

It even came with handwriting built in as authentication mechanism! AI detectors hate this secret! On a more serious note - US removed cursive from their curriculum almost two decades ago - something i cant wrap my head around as cursive is something the rest of the world(?) uses starting in middle school and onwards through the whole adult life.

I don't know what the rest of the world calls "cursive", but here in the US the cursive we get taught is strictly inferior: slower to write, less compact on the page, and harder to read (while also being strictly uglier than true calligraphy). It's a script designed for allowing you to avoid lifting a quill from the page and thereby avoiding ink blots; it's entirely obsolete.

Re: Do AI detectors work? Students face false cheating accusations

#466

Post by BubbleRings »

Let's talk about the actual one page extract of her essay, which can be seen in the article, it is the second image.

My take is, if she used AI to generate that, she didn't use a very good one. I don't think ChatGPT would make the grammar and clarity mistakes that you see in the image text.

I see this:

"should be exposed to many of these forms and models to strengthen understanding" - much better as "should be exposed to as many of these forms and models as possible to strengthen their understanding"

"it is mentioned that students should have experiencing understanding the..." - plainly wrong, better would be "it is mentioned that students should have experience understanding the..."

"time with initial gird models" -> "time with initial grid models"

And there are other lines that could be improved.

My opinion is, the only solution to this problem is to allow AI detectors to flag work, but that when a work is flagged, that flagging just triggers a face to face meeting between the student and the professor, where the student is required to show through discussion of the work that they understand it well enough to have written it.

However! Often the professor is too busy, or isn't smart enough to review the writing of the student carefully enough to determine whether the student really wrote it. What to do? Why of course: invent AI systems that are really good at interviewing students well enough to tell if they really wrote a piece of work. Yeah you laugh but it will happen some day soon enough.

Re: Do AI detectors work? Students face false cheating accusations

#467

Post by tropdrop »

k__ wrote: post #165

I'm a professional writer and test AI and AI detectors ever other month. Plagiarism detectors kinda work, but you can always use one to locate plagiarized sections and fix them yourself. I have a plagiarism rate under 5%, usually coming from the use of well known phrases. An AI usually has over 10%. Obviously that doesn't help in an academic context when people mark their citations. The perplexity checks don't work,…

FWIW Turnitin does treat things like quotes and footnotes a little differently on the academic side – on the instructor end, it simply gives you an estimation of the amount of text it found that has appeared somewhere else. Citations usually account for about 5-10% "potentially plagiarized" but anything below 10% is treated as fine by the software. You can always go check each of the sections and see if it's a quote or not; if you have a paper that consists of more than 10% quotes it's not a good paper anyway and should be revised.

I did have a very interesting case once of a student who copied and pasted someone's Master's thesis for sections of her paper, but also listed that thesis in the citations... it remains up to the jury (not me) to decide whether she just didn't understand what plagiarism was. I would not have known if Turnitin didn't mark it as 30% plagiarized.

Disclaimer: Someone more senior than I was in charge of the decision to use this software, but it was interesting to see it in action

Re: Do AI detectors work? Students face false cheating accusations

#468

Post by lukan »

ordu wrote: post #422
lukan wrote: post #376

Earlier quoted context omitted.

I hope that was sarcasm?

Probably. I'm not sure myself. It is, because I'm becoming tired with the current AI hype. It lasts too long to be funny. OTOH, professor talking with a student is a good way to assess the academic performance of the student, but there are some caveats beyond costs. For example, professor will struggle to be an objective judge. Moreover even if they succeed, they would face accusations of discrimination in any case.…

"OTOH, professor talking with a student is a good way to assess the academic performance of the student, but there are some caveats beyond costs"

Why not have the testing done externally, by really neutral persons?

But AIs and especially LLMs are way too unreliable for the foreseeable future.

Re: Do AI detectors work? Students face false cheating accusations

#469

Post by bearjaws »

cfcf14 wrote: post #143

AI detectors do not work. I have spoken with many people who think that the particular writing style of commercial LLMs (ChatGPT, Gemini, Claude) is the result of some intrinsic characteristic of LLMs - either the data or the architecture. The belief is that this particular tone of 'voice' (chirpy sycophant), textual structure (bullet lists and verbosity), and vocab ('delve', et al) serves and and will continue to se…

I recently deployed an AI detector for a large K12 platform (multi-state 20k+ students), and they DO work in the sense of saving teachers time.

You have to understand, you are a smart professional individual who will try to avoid being detected, but 6-12th grade students can be incredibly lazy and procrastinate. You may take the time to add a tone, style and cadence to your prompt but many students do not. They can be so bad you find the "As an AI assistant..." line in their submitted work. We have about 11% of assignments are blatantly using AI, and after manual review of over 3,000 submitted assignments GPTZero is quite capable and had very few (Do you want teachers wasting time loading, reviewing and ultimately commenting on clear AI slop? No you do not, they have very little time as is and that time will be better spent helping other students.

Of course, you need a process to deal with false positives, the same way we had one for our plagiarism detector. We had to make decisions many years ago about what percentage of false positives is okay, and what the process looks like when it's wrong.

Put simply, the end goal isn't to catch everyone, it's to catch the worst offenders such that your staff don't get worn down, and your students get a better education.

Re: Do AI detectors work? Students face false cheating accusations

#470

Post by eulenteufel »

In my observation something paradox happens when teachers use LLM-Detectors to fail their students on dubious detection probabilities.

The teacher accuses the student of using the LLM to perform the task they are assigned. This entails not properly understanding the assignment and presenting an accomplishment which has not been achieved by the student themselves.

On the other hand the teacher using an LLM tool also do not understand the reasoning of the decision and present often present them as their own judgement. A judgement that has not truly been felled by the teacher because they do not use the tool for understanding but for deferring their responsibilities.

In doing so the teacher is engaging in the same act of (self-)deception they are accusing the student of: presenting an achievement not truly reached through their own understanding, even if the situation necessitates it (non-deferrable learning vs. non-deferrable decision).

The use of LLM-detection in this way thus mirrors the very problem it seeks to address.

Post reply on HN