Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

471–480 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#471

Earlier quoted context omitted.

I've found that in adulthood, I've still been judged on results, not effort, and unless we're going to drastically reduce student:teacher ratios, I don't see how you even could judge on effort. Some kids are going to learn more quickly than others, and for them, no effort will be required. At best you might assign them busywork, but that doesn't take effort just as it wouldn't take effort for an adult to do the work.

I also don't think effort can be recognised in some spaces; as a programmer, I often produce results that in the end, result in very few lines of code written, looking at the end result alone doesn't indicate much. It's like looking at a hand carved match-stick judging the result as low effort, not knowing that they started with a seed.

The end result is never the code itself. In fact, the end result exists over time, and often the shape of the result in the time dimension is better the shorter the code and the more thorough the intangible forethought.

But yes, I don’t know how clear must I be about it—this is learning (for very young humans still psychologically immature), that’s exactly why it has to be spelled out that evaluation must be on the effort, precisely because it is never on the effort in any other activity in adulthood.

Re: Do AI detectors work? Students face false cheating accusations

#472

Earlier quoted context omitted.

I agree. There are small question about bias (gender, race) etc in these oral systems, but I think they are resolvable and much better than written essays (which are now written by AI).

the teacher knows you either way so the bias would be there for the written exam as well

In a written exam they can cover the names - give you a random number as you enter the room and you write that on the paper, and but your name and number on a different paper. You also need to type everything out on a computer with spell check. (and even then if you write bucket or pail will identify you but it is unlikely any professor knows you well enough to tell those)

When you audition for a symphony you perform behind a curtain and are required to wear soft slippers (so they can't tell if you are a wearing high heals - female).

We can probably use voice changers so the examiner cannot tell who you are by your voice, but those tend to be fatiguing.

Re: Do AI detectors work? Students face false cheating accusations

#473
post #338
post #330

Earlier quoted context omitted.

Written assay evaluation is not and has never been an effective evaluation. It was always a cost saving measure because allocating 30min face to face time with each individual student for each class is such a gigantic cost for the institution that they cannot even imagine doing it. Think about that the next time you look at your student debt, it couldn’t even buy you 30min time per class individually with the teacher…

There is truth to this perspective but it's also missing one of the fundamental purposes of writing essays in an educational setting. Writing essays isn't just about evaluation, it's also about teaching you how to think. The process of reading textual material, thinking about it, and then producing more textual material about the stuff you just read (and maybe connecting it to other stuff that you've read in the past…

All excellent point, but I'd like to add that it also forces you to do your own research the correct way, by surveying the current state of academic research and then finding and incorporating scholarly sources into your own arguments. Every academic essay I ever wrote after high school started with a trip to the library and JSTOR. I had to guide my own education instead of learning from the teacher and then repeating what had been taught.

Re: Do AI detectors work? Students face false cheating accusations

#474

One of my kid's teachers sent out a warning to students that all essays would be checked with AI detection software and the repercussions one would face if caught. A classmate did an AI check on the teacher's warning and it came back positive for having been AI-generated.

While I understand the spirit of your message, you should not care about that.

"One of my kid's teachers set out a warning to students that all essays would be checked against the other students' essays to see if they are the same and the repercussions one would face if caught. A classmate did a Google search and found the questions of the essay as examples on a book."

One thing is perfectly valid, the other one is not.

Then of course, there are shades of gray. Using ChatGPT for some things is not copying and you can even say the kids are learning to use the tool, but if you use it for 95% of the essay, it is.

Re: Do AI detectors work? Students face false cheating accusations

#475

One of my kid's teachers sent out a warning to students that all essays would be checked with AI detection software and the repercussions one would face if caught. A classmate did an AI check on the teacher's warning and it came back positive for having been AI-generated.

Hah, that’s great! Hopefully this dramatic chapter in history is a short one, and we learn to adapt away from graded homework. A 4% false positive rate is insane when that could mean failure and/or expulsion, and even more so when any serious cheater can get around in two minutes with a “write in the style of…” preprompt.

Re: Do AI detectors work? Students face false cheating accusations

#476
post #474

One of my kid's teachers sent out a warning to students that all essays would be checked with AI detection software and the repercussions one would face if caught. A classmate did an AI check on the teacher's warning and it came back positive for having been AI-generated.

While I understand the spirit of your message, you should not care about that. "One of my kid's teachers set out a warning to students that all essays would be checked against the other students' essays to see if they are the same and the repercussions one would face if caught. A classmate did a Google search and found the questions of the essay as examples on a book." One thing is perfectly valid, the other one is n…

Hmm I think you may have misinterpreted. The accusation isn’t that the teacher used AI, the accusation is that these tools are unreliable

Re: Do AI detectors work? Students face false cheating accusations

#477
post #205

Earlier quoted context omitted.

> I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us. That basically never happens and the outcome is the result of some sort of struggle. Usually just a peaceful one in the courts and legislatures and markets, but a struggle nonetheless. > new methods should be used to assess student performance. Such as? We need an answer now because students are bei…

> Such as? We need an answer now because students are being assessed now. Two decades ago, when I was in engineering school, grades were 90% based on in-person, proctored, handwritten exams. So assignments had enough weight to be worth completing, but little enough that if someone cheated, it didn't really matter as the exam was the deciding factor. > Return to the old "viva voce" exam? Still used for PhDs. But that…

I have 53 students in my class right now. A 15-minute oral exam works out to 13.25 hours of exam time, assuming perfect efficiency. As a comparison, our in-class time (3 hours over 16 weeks) works out to only about 48 hours. So a single oral exam works out to 1/4th of all class time.

But in principle this is not a problem for me, I already spend at least this much time grading papers, and an oral exam would be much more pleasant. The real problems will come up when (1) students are forced to schedule these 15-minute slots, and (2) they complain about the lack of time and non-objective grading rubric.

Re: Do AI detectors work? Students face false cheating accusations

#478
post #438

Earlier quoted context omitted.

When I studied physics at Exeter University they still used the tutorial system and finals. Tutorials were held fortnightly; the tutorial groups were typically three or four students. There was no obligation to turn up to lectures or even tutorials. You just had to pass the end of year exams to be allowed to continue to the final. The class of degree that was awarded depended on the open note final exam and the repor…

What about those of us who can explain our ideas and thinking clearly and in great detail in writing but would struggle to even prove we've heard of the topic orally?

you could ask for reasonable accommodations - e.g. if you have a recognised medical condition, or even just going through a rough time - e.g. ask to be allowed to write down your answer while they wait.

Re: Do AI detectors work? Students face false cheating accusations

#479
post #476
post #474

Earlier quoted context omitted.

While I understand the spirit of your message, you should not care about that. "One of my kid's teachers set out a warning to students that all essays would be checked against the other students' essays to see if they are the same and the repercussions one would face if caught. A classmate did a Google search and found the questions of the essay as examples on a book." One thing is perfectly valid, the other one is n…

Hmm I think you may have misinterpreted. The accusation isn’t that the teacher used AI, the accusation is that these tools are unreliable

Then I completely agree hahaha

Re: Do AI detectors work? Students face false cheating accusations

#480
post #438

Earlier quoted context omitted.

When I studied physics at Exeter University they still used the tutorial system and finals. Tutorials were held fortnightly; the tutorial groups were typically three or four students. There was no obligation to turn up to lectures or even tutorials. You just had to pass the end of year exams to be allowed to continue to the final. The class of degree that was awarded depended on the open note final exam and the repor…

What about those of us who can explain our ideas and thinking clearly and in great detail in writing but would struggle to even prove we've heard of the topic orally?

In real life you need to be able to communicate written, in formal talks, and in informal discussions.

Those of you who severely lack any of the three will be penalized. Just like someone who can discuss a topic orally but could not write it up would be penalized.

Post reply on HN