Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

571–580 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#571
Financial institutions utilize zero trust principles in order to combat financial cheating.

Learning centers need to adopt similar principles in order to avert overt homework and exam cheating.

Do we trust the students, or the professors? No.

So why continue to treat them as if we did?

Re: Do AI detectors work? Students face false cheating accusations

#572
The LLMs de-value the viability of homework, and assignments consisting of at-home busywork. As an alternative, teachers will have to put more emphasis on proctored exams.

I say good riddance, that's exactly how it should be. At-home busywork is a scourge on especially K-12 students. Yet, every teacher has been loading their students up with homework, because that's their idea of what a "good teacher" is supposed to do.

The faster technology overcomes this problem, the better.

Re: Do AI detectors work? Students face false cheating accusations

#573

Earlier quoted context omitted.

> The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. In Germany, all exams are like this. Homework assignments are either just a prerequisite for taking exam but the grade is solely from the exam, or you may get some small point bonus for assignments/projects. > But any teacher doing this will end up with no students signing up for…

Germany isn't special, (almost) all exams work like that in the US as well. I don't know why he was implying otherwise. Almost all degrees have required courses in the US as well. You point to a true failure in incentives. And yet, the US has the highest density of renowned universities.

For online courses it is no longer the case that exams are proctored in person. Most higher education in the United States is done at community colleges and regional state universities.

Re: Do AI detectors work? Students face false cheating accusations

#574

Earlier quoted context omitted.

> The main courses are mandatory in order to obtain the degree. You can't "not sign up" for linear algebra if it's in your curriculum. The course might be mandatory but which professor you choose isn't. What if multiple professors teach it? Word gets around and everyone chooses the easy profs.

In Germany, there's no such choice. There are no competing alternative courses that can substitute for each other, the very thought seems rather strange. There is one Linear Algebra course. You have to pass it to get your degree. Typically, it's taught by the same prof for many years, but it might also rotate between different chairs and profs (but only one in each semester and the "design" and requirements of the co…

It seems more strange in my opinion that you'd never have a course thats popular enough that more than one teacher holds sessions for it.

You don't have the choice to not take the class, you just have choice with which professor you would like to take it with. And often you would have to get lucky anyway, since that session may be filled so you'd have to take it with the "harder" teacher anyway.

For example with the popularity of computer science and STEM in general, at my school there were often 2-3 teachers teaching linear algebra in any given semester. And same for popular classes like calculus or introductory physics. Students would often lookup online which teacher was considered easier, but they still had to take the class.

Re: Do AI detectors work? Students face false cheating accusations

#575
post #448
post #265

Earlier quoted context omitted.

That depends on the questions. There are also open book exams. A viva is a type of exam so I don’t see they are incompatible with assessing research

Not every class is STEM. Are you writing a 4000 word research paper sitting in class?

No, but we had to write essays in class during exams.

There's a good question about the future and utility of long at-home research paper projects in school, but it's not a cornerstone of education.

In 9th grade I procrastinated the semestral paper so much that I bought an essay online that explored unexpected gay themes in Ray Bradbury's corpus of work. I was so lazy I didn't even read it first, only skimmed it, and then back to Runescape. So it's not like this is a new problem due to LLMs, and I think take-home semester projects are all quite bad for these reasons that predate LLMs.

(It turned out to be such a phenomenally audacious essay that my teacher started fascinated email correspondence with me about it and I was forced to not only study the essay but also read the quoted parts of his work. Ugh, backfire.)

Re: Do AI detectors work? Students face false cheating accusations

#576

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

I remember taking a math class in college and the professor had a very unique way of dealing with cheating. He let us use our books, notes, and "any calculator capability" from our TI-84's. His rationale is that students will try to use these tricks anyways so just let them and then update the test to be "immune" from these advantages. Before every test he mentioned that we could use all those tools but always said "but please study, your books, notes and calculators won't save you".

Long term I see education going this route, rather than preventing students from using AI tools, update course curriculum so that AI tools don't give such an advantage.

Re: Do AI detectors work? Students face false cheating accusations

#577
post #319

Earlier quoted context omitted.

I think you're far more optimistic than I am. I think that we'll see fundamental changes, but it will be based on cheaper consumer goods because all of the back end white collar labor that adds costs to them will be (for all intents and purposes) free. But we will see the absolute destruction of the middle class. This will be the death blow. The work week will change, but only because even more people will work multi…

Have you seen the film Zardoz? Looking back on it, I think it could be weirdly precient. Two classes of society; one living a life of leisure, the other fighting on the plains. (.. maybe minus Connery in a mankini)

That's a very common theme in literature concerning the future of society when technology and social hierarchy are applied ad absurdum.

The Time Machine is a very famous example.

Re: Do AI detectors work? Students face false cheating accusations

#578

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

> The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. In Germany, all exams are like this. Homework assignments are either just a prerequisite for taking exam but the grade is solely from the exam, or you may get some small point bonus for assignments/projects. > But any teacher doing this will end up with no students signing up for…

> This is because universities are paid from tax money in Germany and most of Europe.

Almost every university in the US takes federal money and relies on federal loan guarantees to keep the high revenues pumping through. In exchange, the schools are subject to requirements by the government and they impose many. I think the bigger issue is the size and scope of higher ed here and if it's actually a good idea to to tell every school how to run their exams (and enforce it).

Re: Do AI detectors work? Students face false cheating accusations

#579
post #507

Earlier quoted context omitted.

The problem is that AI detection is far closer to 0% than 100%,. It's really bad and the very nature of this tech makes it impossible to be good.

As someone working in this field, it is simply not closer to 0% People keep using these "gotcha" examples and never actually look at the stats for it. I get it, there are some terrible detectors out there, and of course they are the free ones :) https://edintegrity.biomedcentral.com/articles/10.1007/s4097... GPTZero was correct in most scenarios where they used basic prompts, and only had one false positive. We did a…

> https://edintegrity.biomedcentral.com/articles/10.1007/s4097...

> GPTZero was correct in most scenarios where they used basic prompts, and only had one false positive.

One false positive out of only "five human-written samples", unless I'm misreading.

Say 50 papers are checked, with 5 being generated by AI. By the rates of GPTZero in the paper, 3 AI-generated papers would be correctly flagged and 9 human-written papers would incorrectly flagged. Meaning a flagged paper is only 25% likely to actually be AI-generated.

Realistically the sample size in the paper is just far too small to make any real conclusion one way or another, but I think people fail to appreciate the difference between false positive rate and false discovery rate.

Re: Do AI detectors work? Students face false cheating accusations

#580

I don't understand most of the comments here. I couldn't cheat in high school because we couldn't use our phones during class. Not for worksheets nor quizzes and especially not exams whether they be multiple choice, oral, or essays. Yet the top threads here act like we need a whole refactor of schooling, many people suggesting we rely on viva voce exams and proctored exams. What exactly do you think that's solving ov…

Did your high school not have any kind of summative homework?

In many places, particularly in the U.S., there are few invigilated exams, and quite a lot of your overall grade will be comprised of coursework. This, combined with the inexorable advance of digitalisation of education has led to where we are now.

Certainly once you get to university level, there are projects which simply take too long to be done in the classroom, such as a dissertation or final report. These projects have always been vulnerable to commissioning rather than plagiarism, and you’d be appalled to realise how common it actually is even in higher prestige places. LLMs have simply lowered that bar to make it even more common.

This is a genuine problem, and people are more sophisticated cheaters than you might initially think.

Post reply on HN