Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

Post reply on HN
Print view View original

331–340 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#331

Post by cryptonym »

johnisgood wrote: post #324
red_admiral wrote: post #278

Earlier quoted context omitted.

> It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. It never was. It's just even more ineffective now that AI exists, than before. The central example of this is college admissions statements. Some kids have the advantage both of parents who can afford to give them the experiences that look good on such an essay (educational trips to Africa, lessons in two…

It is not so black and white though: there is a difference between having your whole essay written by a tutor, or having some things corrected by the tutor, or the tutor giving you general tips that you yourself apply.

Just like there is a difference between having your whole essay written by a LLM, or having some things corrected by the LLM, or the LLM giving you general tips that you yourself apply.

Re: Do AI detectors work? Students face false cheating accusations

#332

Post by abenga »

kubb wrote: post #300
light_hue_1 wrote: post #277

Earlier quoted context omitted.

Oh yes. When I'm teaching a class of 200 students it's totally plausible that we're going to do 10 15 minute one on one conversations with every student. Because that's only 20 days non stop with no sleep. We would need to increase the amount of teaching staff by well over 10x to do this. The costs would be astronomical.

But you can read 200 essays? At this point you can be replaced with AI, you’re not adding any value anymore.

Essays are async and easier to delegate.

Re: Do AI detectors work? Students face false cheating accusations

#333

Post by piltdownman »

pjc50 wrote: post #205
lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

> I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us. That basically never happens and the outcome is the result of some sort of struggle. Usually just a peaceful one in the courts and legislatures and markets, but a struggle nonetheless. > new methods should be used to assess student performance. Such as? We need an answer now because students are bei…

Viva Voce scales fine if AI is the invigilator!

Re: Do AI detectors work? Students face false cheating accusations

#334

Post by johnisgood »

cryptonym wrote: post #331
johnisgood wrote: post #324

Earlier quoted context omitted.

It is not so black and white though: there is a difference between having your whole essay written by a tutor, or having some things corrected by the tutor, or the tutor giving you general tips that you yourself apply.

Just like there is a difference between having your whole essay written by a LLM, or having some things corrected by the LLM, or the LLM giving you general tips that you yourself apply.

I agree.

Re: Do AI detectors work? Students face false cheating accusations

#335

Post by lwhi »

delusional wrote: post #199
lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

> I remember being told at school, that we weren't allowed to use calculators in exams I remember being told the same thing, but I happen to believe that it was a fantastic policy, with a lackluster explanation. The idea that you wouldn't have a calculator was obviously silly, even at the time, but underlying observation that relying on the calculator would rob you of the mental exercise the whole ordeal was supposed…

Great point .. I agree; education is fundamentally exercise for the brain. Without challenge, the 'muscle' can't develop.

I especially agree that essay writing is hugely useful. I'd even go as far as saying, the ability to think clearly is fundamental to a happy life.

Re: Do AI detectors work? Students face false cheating accusations

#336

Post by tonypace »

strogonoff wrote: post #201
lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

The best method for assessing performance when learning is as old as the world: assess the effort, not how well the result complies with some requirements. If the level of effort made is high, but the outcome does not comply in some way, praise is due. If the outcome complies, but the level of effort is low, there is no reason for praise (what are you praising? mere compliance?) and you must have set a wrong bar. Not…

It's fairly simple in most situations. If it doesn't involve a computer, it's handwritten in class. If it does involve a computer, it's a temporarily offline computer. We have figured out solutions to these problems already.

Re: Do AI detectors work? Students face false cheating accusations

#337

Post by ben_w »

lotsofpulp wrote: post #291
lwhi wrote: post #267

Earlier quoted context omitted.

> Personally I think all this is unpredictable and destabilizing. I completely agree, but then again it seems to me that society also functions according to many norms that were established due to historical context; and could / should be challenged and replaced. Our education system was based on needs of the industrial revolution. Ditto, the structure of our working week. My bet: We will see our working / waking liv…

>Fundamental questions like 'why do we work', 'what do we need' and 'what do we want' will be necessarily brought to the fore. All the low paid, physically laborious work is not affected by AI, so there will be plenty of work, especially with aging populations around the world. The question is will it be worth doing (can the recipients of the work pay enough) without being able to provide the dream of being able to o…

> All the low paid, physically laborious work is not affected by AI, so there will be plenty of work, especially with aging populations around the world.

Low paid physically labourorious work has been increasingly automated since the wheel started helping potters.

The history of computers often ends up referencing Jacquard looms because it used punched cards.

LLMs have been demonstrated controlling various robots.

AI's coming for all jobs, it's just a question of which jobs cease first.

Re: Do AI detectors work? Students face false cheating accusations

#338

Post by Teever »

gklitz wrote: post #330
lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

Written assay evaluation is not and has never been an effective evaluation. It was always a cost saving measure because allocating 30min face to face time with each individual student for each class is such a gigantic cost for the institution that they cannot even imagine doing it. Think about that the next time you look at your student debt, it couldn’t even buy you 30min time per class individually with the teacher…

There is truth to this perspective but it's also missing one of the fundamental purposes of writing essays in an educational setting. Writing essays isn't just about evaluation, it's also about teaching you how to think.

The process of reading textual material, thinking about it, and then producing more textual material about the stuff you just read (and maybe connecting it to other stuff that you've read in the past) is a critical way of developing thinking skills and refining your ability to communicate to an audience.

The value of that shouldn't be overlooked just like the value of basic numeracy shouldn't be overlooked because we all carry calculators.

You're right that it would be better if post secondary institutions would test people's ability to think in more ways than just what they can regurgitate onto a piece of paper, if only because that can be easily cheated but that doesn't mean that there isn't personal benefit in the experience of writing an essay.

I may not be the best writer but I am a better writer because I wrote essays in university, and I may not be great at math but I can reason and estimate about a variety of things because I have taken many math courses. These things have ultimately made me a better thinker and I am grateful to have had that imparted to me.

Re: Do AI detectors work? Students face false cheating accusations

#339

Post by lm28469 »

lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

In France my essays were written in class, no phones, no book, just your brain, a sheet a paper and a pen. That's still 100% doable today

Re: Do AI detectors work? Students face false cheating accusations

#340

Post by mrweasel »

viraptor wrote: post #282
mrweasel wrote: post #184

The part that annoys me is that students apparently have no right to be told why the AI flagged their work. For any process where an computer is allowed to judge people, where should be a rule in place that demands that the algorithm be able explains EXACTLY why it flagged this person. Now this would effectively kill off the current AI powered solution, because they have no way of explaining, or even understanding, w…

> kill off the current AI powered solution, because they have no way of explaining That's not correct. Some solution look at perplexity for specific models, some will look at ngram frequencies, and similar approaches. Almost all of those can produce a heatmap of "what looks suspicious". I wouldn't expect any of the detection systems to be like black boxes relying on LLM over the whole text.

Sorry if this is "moving the goal post", but I wouldn't call looking at ngram frequencies for AI. Producing a heatmap doesn't tell you why something is suspicious, but it's obviously better than telling you nothing.

In any case, if you where to use LLMs, or other black box solutions, you'd have to yank those out, if you where met with a requirement to explain why something is suspicious.

Post reply on HN