Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

Post reply on HN
Print view View original

411–420 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#411

Post by krisoft »

anonzzzies wrote: post #124

I don't know what these 'students' are doing, but it's not very hard to prompt a system into not using the easily detectable 'ai generated' language at all. Also adding in some spelling errors and uncapping some words (like ai above here) makes it more realistic. But just adding an example of how you write and telling it to keep your vocabulary and writing some python to post process it makes it impossible to detect…

> I don't know what these 'students' are doing, but it's not very hard to prompt a system into not using the easily detectable 'ai generated' language at all.

Writing their essays by hand. That is what they are doing.

Re: Do AI detectors work? Students face false cheating accusations

#412

Post by high_na_euv »

fuzzy_biscuit wrote: post #395

If AI detection cannot be 100% accurate, I do not believe it is an appropriate solution for judging the futures of millions of students and young people. Time to move on. Either from the tech or from the essay format. In either case, we need to change our standards around mastery of subject matter.

AI sucks, but on the other hand

Judges and police officers arent 100% accurate too

Re: Do AI detectors work? Students face false cheating accusations

#413

Post by School-Cotton »

ClassyJacket wrote: post #127
willvarfar wrote: post #90

Earlier quoted context omitted.

Do you think it stupid to scan kids for weapons, or stupid to think that a metal detector will find weapons?

I think it's stupid to have a country where guns are legal.

If the US were a functional democracy, and continued letting unrestricted gun ownership be legal, you could argue that the US citizenry is being stupid. But the US is not a functional democracy, and meaningfully reforming anything is impossible, regardless of whether most people want it or whether it’s a good idea.

Re: Do AI detectors work? Students face false cheating accusations

#414

Post by caseyy »

lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

We blasted through the “you won’t always have AI in your pocket” phase in a blink of an eye. Local LLMs were running on smartphones before the world got to terms of LLMs being used everywhere. It’s one of many examples of exponential technological advancement.

Re: Do AI detectors work? Students face false cheating accusations

#415

Post by internet101010 »

Ekaros wrote: post #137
anonzzzies wrote: post #124

I don't know what these 'students' are doing, but it's not very hard to prompt a system into not using the easily detectable 'ai generated' language at all. Also adding in some spelling errors and uncapping some words (like ai above here) makes it more realistic. But just adding an example of how you write and telling it to keep your vocabulary and writing some python to post process it makes it impossible to detect…

Sounds like almost same level of effort than actually just writing it yourself. Or getting AI write draft and then just rewriting it quickly. Humans are lazy, students especially so.

Sounds like they would make good programmers.

Re: Do AI detectors work? Students face false cheating accusations

#417

Post by bigfudge »

ninalanyon wrote: post #365
gklitz wrote: post #330

Earlier quoted context omitted.

Written assay evaluation is not and has never been an effective evaluation. It was always a cost saving measure because allocating 30min face to face time with each individual student for each class is such a gigantic cost for the institution that they cannot even imagine doing it. Think about that the next time you look at your student debt, it couldn’t even buy you 30min time per class individually with the teacher…

When I studied physics at Exeter University they still used the tutorial system and finals. Tutorials were held fortnightly; the tutorial groups were typically three or four students. There was no obligation to turn up to lectures or even tutorials. You just had to pass the end of year exams to be allowed to continue to the final. The class of degree that was awarded depended on the open note final exam and the repor…

I agree. There are small question about bias (gender, race) etc in these oral systems, but I think they are resolvable and much better than written essays (which are now written by AI).

Re: Do AI detectors work? Students face false cheating accusations

#418

Post by tonypace »

ben_w wrote: post #241
llmthrow102 wrote: post #186

Earlier quoted context omitted.

You mean, you work with devs who are using AI to generate their code.

Not saying where, but well before transformers were invented, I saw an iOS project that had huge chunks of uncompiled Symbian code in the project "for reference", an entire pantheon of God classes, entire files duplicated rather than changing access modifiers, 1000 lines inside an always true if block, and 20% of the 120,000 lines were: // And no, those were not generally followed by a real comment.

And yet, I have an unfortunately clear mental picture of the human that did this. In itself, that is a very specific coding style. I don't imagine an LLM would do that. Chat would instead take a couple of the methods from the Symbian codebase and use them where they didn't exist. The God classes would merely be mined for more non-existent functions. The true if block would become a function. And the # lines would have comments on them. Useless comments, but there would be text following every last one of them. Totally different styles.

Re: Do AI detectors work? Students face false cheating accusations

#419

Post by caseyy »

lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

The old colloquium exam format reigns supreme again. And that is fantastic. We shouldn’t reserve it for only “most important” occasions because quality education is important enough by itself.

Re: Do AI detectors work? Students face false cheating accusations

#420

Post by bigfudge »

michaelt wrote: post #254
pjc50 wrote: post #205

Earlier quoted context omitted.

> I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us. That basically never happens and the outcome is the result of some sort of struggle. Usually just a peaceful one in the courts and legislatures and markets, but a struggle nonetheless. > new methods should be used to assess student performance. Such as? We need an answer now because students are bei…

> Such as? We need an answer now because students are being assessed now. Two decades ago, when I was in engineering school, grades were 90% based on in-person, proctored, handwritten exams. So assignments had enough weight to be worth completing, but little enough that if someone cheated, it didn't really matter as the exam was the deciding factor. > Return to the old "viva voce" exam? Still used for PhDs. But that…

Interestingly, in the UK strong student preferences against proctored exams and nervousness about how mental health issues interact with exams means universities are resisting dropping coursework, despite everyone knowing that most coursework is ai generated.
Post reply on HN