Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

Post reply on HN
Print view View original

211–220 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#211

Post by sqeaky »

briandear wrote: post #191
immibis wrote: post #151

Earlier quoted context omitted.

[flagged]

“Regularly” is not a particularly accurate word. 50 million K12 students in the U.S. — how many mass murders are “regular?”

More than once a week not quite once a day.

Rather than trying to diminish something that's completely preventable and abhorrent maybe we could discuss ways to actually prevent it. Because this isn't a problem anywhere else so clearly it's preventable.

If AI can be part of a solution here this is a reasonable place to discuss it.

Re: Do AI detectors work? Students face false cheating accusations

#212

Post by A4ET8a8uTh0 »

pjc50 wrote: post #205
lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

> I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us. That basically never happens and the outcome is the result of some sort of struggle. Usually just a peaceful one in the courts and legislatures and markets, but a struggle nonetheless. > new methods should be used to assess student performance. Such as? We need an answer now because students are bei…

I realize that the level of effort for oral exam is greater for both parties involved. However, the fact it does not scale is largely irrelevant in my view. Either it evaluates something well or it does not.

And, since use of AI makes written exams almost impossible, this genuinely seems to be the only real test left.

Re: Do AI detectors work? Students face false cheating accusations

#213

Post by anonzzzies »

Ekaros wrote: post #137
anonzzzies wrote: post #124

I don't know what these 'students' are doing, but it's not very hard to prompt a system into not using the easily detectable 'ai generated' language at all. Also adding in some spelling errors and uncapping some words (like ai above here) makes it more realistic. But just adding an example of how you write and telling it to keep your vocabulary and writing some python to post process it makes it impossible to detect…

Sounds like almost same level of effort than actually just writing it yourself. Or getting AI write draft and then just rewriting it quickly. Humans are lazy, students especially so.

When I look around in the shared open workspace I am in currently for a meeting, everyone (programmers, PR, marketing) has Claude/GPT/Perplexity on their screen. 100% of the people here. So I guess this will not be limited to students.

Re: Do AI detectors work? Students face false cheating accusations

#214

Post by dambi0 »

lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

An essay written under examination conditions is fine. We don’t need new assessment techniques. We have known how to asses that a student and that student alone for centuries.

Re: Do AI detectors work? Students face false cheating accusations

#215

Post by ben_w »

pjc50 wrote: post #205
lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

> I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us. That basically never happens and the outcome is the result of some sort of struggle. Usually just a peaceful one in the courts and legislatures and markets, but a struggle nonetheless. > new methods should be used to assess student performance. Such as? We need an answer now because students are bei…

> Such as? We need an answer now because students are being assessed now.

My current best guess, is to hand the student stuff that was written by an LLM, and challenge them to find and correct its mistakes.

That's going to be what they do in their careers, unless the LLMs get so good they don't need to, in which case https://xkcd.com/810/ applies.

> Personally I think all this is unpredictable and destabilizing. If the AI advocates are right, which I don't think they are, they're going to eradicate most of the white collar jobs and academic specialties for which those people are being trained and evaluated.

Yup.

I hope the e/acc types are wrong, we're not ready.

Re: Do AI detectors work? Students face false cheating accusations

#216

Post by sqeaky »

briandear wrote: post #206
xnorswap wrote: post #150

Earlier quoted context omitted.

Just because something is technically legal, doesn't mean it's in any way common or part of UK culture to own a gun. There hasn't been a school shooting in the UK for nearly 30 years. Handguns were banned after the last school shooting and there hasn't been one since. https://en.wikipedia.org/wiki/Category:School_shootings_in_t... Although that fact is sometimes forgotten by schools who copy the US in having "active…

Yet the murder rate was unaffected by the gun ban.

Why do people say such unsubstantiated nonsense. Places with guns have more death. And it's obvious to see why guns are a tool for for killing, and they're pretty effective.

Re: Do AI detectors work? Students face false cheating accusations

#217

Post by A4ET8a8uTh0 »

UncleMeat wrote: post #197
lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

This is mostly true, but it is also important to recognize that “hey just invent a new evaluation methodology” is a rough thing to ask people to do immediately. People are trying to figure it out in a way that works.

Sadly, this is not what is happening. Based on the article ( and personal experience ), it is clear that we tend to happily accept computer output as a pronouncement from the oracle itself.

It is new tech, but people do not treat it as such. They are not figuring it out. Its results are already being imposed. It is sheer luck that the individual in question choose to fight back. And even then it was only a partial victory:

"The grade was ultimately changed, but not before she received a strict warning: If her work was flagged again, the teacher would treat it the same way they would with plagiarism."

Re: Do AI detectors work? Students face false cheating accusations

#218

Post by sersi »

A4ET8a8uTh0 wrote: post #212
pjc50 wrote: post #205

Earlier quoted context omitted.

> I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us. That basically never happens and the outcome is the result of some sort of struggle. Usually just a peaceful one in the courts and legislatures and markets, but a struggle nonetheless. > new methods should be used to assess student performance. Such as? We need an answer now because students are bei…

I realize that the level of effort for oral exam is greater for both parties involved. However, the fact it does not scale is largely irrelevant in my view. Either it evaluates something well or it does not. And, since use of AI makes written exams almost impossible, this genuinely seems to be the only real test left.

> And, since use of AI makes written exams almost impossible

Isn't it easy to prevent students from using an AI if they are doing the exams in a big room? I mean when I was a student, most of my exams were written with just access to notes but no computers. Not that much resources needed to control that...

Re: Do AI detectors work? Students face false cheating accusations

#219

Post by ben_w »

mrweasel wrote: post #184

The part that annoys me is that students apparently have no right to be told why the AI flagged their work. For any process where an computer is allowed to judge people, where should be a rule in place that demands that the algorithm be able explains EXACTLY why it flagged this person. Now this would effectively kill off the current AI powered solution, because they have no way of explaining, or even understanding, w…

> For any process where an computer is allowed to judge people, where should be a rule in place that demands that the algorithm be able explains EXACTLY why it flagged this person.

This is a big part of GDPR.

Re: Do AI detectors work? Students face false cheating accusations

#220

Post by ClumsyPilot »

krick wrote: post #58

That's kinda nuts how adult people learned to trust some random algorithms in a year or two. They don't know how it works, they cannot explain it, they don't care, it just works. It's magic. If it says you cheated, you cheated. You cannot do anything about it. I want to emphasize, that this isn't really about trusting magic, it's about people nonchalantly doing ridiculous stuff nowdays and that they aren't held accou…

> They don't know how it works, they cannot explain it, they don't care, it just works. It's magic. If it says you cheated, you cheated. You cannot do anything about it.

People trust a system because other people trust a system.

It does not matter if the system is the inquisition looking for witches, machine or Gulag from USSR.

The system said you are guilty. The system can’t be wrong.

Kafka is rolling in his grave.

Post reply on HN