Do AI detectors work? Students face false cheating accusations • Hacker News Board

Re: Do AI detectors work? Students face false cheating accusations

#731

Post by Quillbert182 » Mon, Oct 21, 2024, 5:31 PM UTC

whatnow37373 wrote: post #714

Use AI to interview students. A conversation is more informative than a static written essay. With LLMs this is now starting to become a possibility. Turn the tables!

One of my professors has started doing that, and I would much rather write an essay. I hate interacting with the chatbot. It feels unnatural, and the chatbot tends to drag out the conversations far longer than they need to go. Ideally, they would have students have conversations with TAs instead, which is what several of my classes have done to great success.

Re: Do AI detectors work? Students face false cheating accusations

#732

Post by mlsu » Mon, Oct 21, 2024, 5:32 PM UTC

skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

I did a "hard" degree and saw classmates who worked half as hard sail by me, because they cheated. Groups that share answer banks, in-class quizzes with answers shared (when they were not supposed to be), group projects that used last year's stuff. All of it, all the way through final exams, which people had answer keys to. I had a few classmates that were formally investigated for cheating by the university; their punishment is to re-take the class -- the cheat's cumulative 3.8 is turned into a 3.75, that's sure to dissuade them from doing it again!

When I tell people that I never cheated, ever, in any class, through my entire degree, I get mostly surprise. You never? Not once?

But I paid for it, I think. Because it was not easy finding a first position out of school -- I certainly got filtered by GPA. It actually enrages me. What is the point of a degree? What exactly is the point of this thing, if most of the signal is false? Why did I work so hard?

Not even to mention -- many of my classmates (about 1 in 5, one in 6 or so?) were granted "accommodations" which granted them twice as much time to take their exams. There are online services: pay $80, get a letter certifying your ADHD, that you can give the school to get these accommodations. It's completely ridiculous.

Re: Do AI detectors work? Students face false cheating accusations

#733

Post by albrewer » Mon, Oct 21, 2024, 5:32 PM UTC

crooked-v wrote: post #159

dot5xdev wrote: post #83

Earlier quoted context omitted.

Moving everything in class seems like a good idea in theory. But in practice, kids need more time than 50 minutes of class time (assuming no lecture) to work on problems. Sometimes you will get stuck on 1 homework question for hours. If a student is actively working on something, yanking them away from their curiosity seems like the wrong thing to do. On the other hand, kids do blindly use the hell out of ChatGPT. It…

> If you don't do homework, then you don't get practice, and you fail the two exams. I'd be cautious about that, because it means the kids with undiagnosed ADHD who are functionally incapable of studying without enforced assignments will just completely crash and burn without absorbing any of the material at all. Or, at least, that's what happened to me in the one and only pre-college class I ever had where "all work…

I completed college with unmanaged ADHD (diagnosed 10 years later; worst result my psych had ever seen on the TOVA lol).

My second and third semesters went exactly as you described for courses where I was exposed to new things and wasn't just repeating high school - mainly because I had no training or coping mechanisms for learning under that type of pedagogy.

After getting my ass kicked in exams and failing a class for the first time in my life, I finally grokked that optional homework assignments were the professor's way of communicating learning milestones to us, and that even though the professor said they weren't graded (unless you asked), you still had to do them or you wouldn't learn the material well enough to pass the exam.

Still had a few bad grades because of the shit foundation I built for myself, but I brought a 2.2 GPA up to a 3.3 by the end.

The point is that it takes is exposure to that style of teaching before it can really be effective.

Re: Do AI detectors work? Students face false cheating accusations

#734

Post by dragonwriter » Mon, Oct 21, 2024, 5:34 PM UTC

sqeaky wrote: post #234

retaJl wrote: post #207

I'm surprised at the number of comments that give up and say that "AI" is here to stay. I'm also surprised that academics rely on snake oil software to deal with the issue. Instead, academics should unite and push for outlawing "AI" or make it difficult to get like cigarettes. Sometimes politicians still listen to academics. It is probably not going to happen though since the level of political apathy among academics…

I can't think of a single time that we've ever willingly put down a technology that a single person could deploy and appear to be highly productive. You may as well try to ban fire. Looking at some of the most successful historical pushbacks against technology, taxes and compensation for displaced workers is about as much as we can expect. Even trying to put restrictions on AI is going to be very practically challeng…

> But I think the most basic of restrictions like mandating watermarks or tracing material of some kind in it might be possible and really that might do a lot to mitigate the worst problems.

Watermarking output (anything that is detectable that is part of the structure of the text, visual--if even imperceptible--image, or otherwise integrated into whatever the primary output is) will make it take a bit more effort to conceal use, but people and tooling will adapt to it very quickly. Tracing material distinct from watermarking -- i.e., accompanying metadata that can be stripped without any impact to the text, image, or whatever else is the primary output -- will do the same, but be even easier to strip, and so have less impact.

Re: Do AI detectors work? Students face false cheating accusations

#735

Post by dsv3099i » Mon, Oct 21, 2024, 5:38 PM UTC

onlypassingthru wrote: post #725

pj_mukh wrote: post #669

Earlier quoted context omitted.

Serious question from someone who is regularly tasked with hiring Juniors. What IS a good assessment for entry-level/right out of college positions? -> GPA can be gamed, as laid out. -> Take Home assessments can mostly be gamed, I want to assess how you think, now which tools you use. -> Personality tests favor the outgoing/extroverts -> On-location tests/leet code are a crapshoot. What should be best practice here?…

Most of the big professional sports already have this figured out. New college graduates have to compete for a spot at training camp. Hire them as temp contracts for two weeks to two months and let them play with the starting team.

Sounds roughly like an internship

Re: Do AI detectors work? Students face false cheating accusations

#736

Post by Oras » Mon, Oct 21, 2024, 5:39 PM UTC

red_admiral wrote: post #278

lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

> It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. It never was. It's just even more ineffective now that AI exists, than before. The central example of this is college admissions statements. Some kids have the advantage both of parents who can afford to give them the experiences that look good on such an essay (educational trips to Africa, lessons in two…

Well, 15 years ago when I did masters, there was a service that would write the essay for you to score A.

Re: Do AI detectors work? Students face false cheating accusations

#737

Post by psunavy03 » Mon, Oct 21, 2024, 5:41 PM UTC

arcbyte wrote: post #723

pj_mukh wrote: post #669

Earlier quoted context omitted.

Serious question from someone who is regularly tasked with hiring Juniors. What IS a good assessment for entry-level/right out of college positions? -> GPA can be gamed, as laid out. -> Take Home assessments can mostly be gamed, I want to assess how you think, now which tools you use. -> Personality tests favor the outgoing/extroverts -> On-location tests/leet code are a crapshoot. What should be best practice here?…

Passion. Juniors you want to hire will have a side project. That's all you need to see.

If someone is willing to do their job well for a fair wage, why do you insist that they make their job their entire life outside work?

Re: Do AI detectors work? Students face false cheating accusations

#738

Post by auggierose » Mon, Oct 21, 2024, 5:45 PM UTC

ckastner wrote: post #226

ben_w wrote: post #219

Earlier quoted context omitted.

> For any process where an computer is allowed to judge people, where should be a rule in place that demands that the algorithm be able explains EXACTLY why it flagged this person. This is a big part of GDPR.

Indeed. Quoting article 22 [1]: > The data subject shall have the right not to be subject to a decision based solely on automated processing [...] [1]: https://gdpr.eu/article-22-automated-individual-decision-mak...

So if an automated decision happens, and the reviewer looks for a second at it, and says, good enough, that will be OK according to GDPR. Don't see what GDPR solves here.

Re: Do AI detectors work? Students face false cheating accusations

#739

Post by dsv3099i » Mon, Oct 21, 2024, 5:46 PM UTC

busyant wrote: post #716

skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

> The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. I completely agree, but the entire higher ed system is moving to on-line instruction. Basically, if the University of follows your suggestion, all of the competing institutions will eat their lunch by offering on-line courses with the "convenience" of on-line assessments" and the…

Depends. If the competing universities degrade into glorified coding boot camps they’ll probably get thier lunch eaten in turn. And graduates need to be getting reasonable job offers as well.

Re: Do AI detectors work? Students face false cheating accusations

#740

Post by jessekv » Mon, Oct 21, 2024, 5:50 PM UTC

skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

> Students are not used to doing the necessary work to learn. They are used to doing the necessary work to pass.

I'd like to point out this has nothing to do with cheating. Cheating happens at all levels of academic performance.

I have not been in university for a while, but I do remember that it was rare that I did my best work for any individual class.

For me it was more of a "satisficing" challenge, and I had to make hard choices about which classes I would not get A's in.

I'm sure some professors might have interpreted my performance in their class as indicative of my overall abilities. I'm fine with that. I learned as much as I could, I maxed out my course load, and I don't regret it at all.