Do AI detectors work? Students face false cheating accusations • Hacker News Board

Re: Do AI detectors work? Students face false cheating accusations

#501

Post by lupusreal » Mon, Oct 21, 2024, 1:40 PM UTC

lupire wrote: post #431

thechao wrote: post #378

Earlier quoted context omitted.

If I'm paying 30k$/yr the professor is damn well reading my essay. If they don't want to teach & grade, they can get a pure research position. Fun fact: pure research positions don't pay as well.

Pure taching positions pay barely minimum wage. Look up "adjunct".

If their situation is that bad they can walk into a local staffing agency and get a factory job that pays 3x the federal minimum wage. Poor pay as a adjunct is a situation they choose for themselves for some reason.

Re: Do AI detectors work? Students face false cheating accusations

#502

Post by lolinder » Mon, Oct 21, 2024, 1:40 PM UTC

kibwen wrote: post #465

junaru wrote: post #351

Earlier quoted context omitted.

It even came with handwriting built in as authentication mechanism! AI detectors hate this secret! On a more serious note - US removed cursive from their curriculum almost two decades ago - something i cant wrap my head around as cursive is something the rest of the world(?) uses starting in middle school and onwards through the whole adult life.

I don't know what the rest of the world calls "cursive", but here in the US the cursive we get taught is strictly inferior: slower to write, less compact on the page, and harder to read (while also being strictly uglier than true calligraphy). It's a script designed for allowing you to avoid lifting a quill from the page and thereby avoiding ink blots; it's entirely obsolete.

This is a bit of an exaggeration.

I learned cursive, then reverted to print, but when I entered a phase of my life where I needed to write several pages a day I quickly went back to (a custom variant of) cursive because it was faster to write in a legible way than print.

When I rush print it quickly becomes illegible. When I rush my cursive it doesn't look quite as nice as it does when I'm writing steadily, but I can still read what I wrote ten years later.

From what I can tell it works because cursive letters are defined in a shape that lends itself to a quick moving pen. Once you learn that shape (both to write and read), you can quickly get words down on a page and then understand them later. If you just try to slur your print in an unprincipled way your letters distort in ways that make them harder to tell apart.

Now, I imagine someone could develop a slurred print that doesn't have connections between letters, but I'd probably call that a cursive anyway.

Re: Do AI detectors work? Students face false cheating accusations

#503

Post by ndriscoll » Mon, Oct 21, 2024, 1:41 PM UTC

strogonoff wrote: post #444

ndriscoll wrote: post #437

Earlier quoted context omitted.

I've found that in adulthood, I've still been judged on results, not effort, and unless we're going to drastically reduce student:teacher ratios, I don't see how you even could judge on effort. Some kids are going to learn more quickly than others, and for them, no effort will be required. At best you might assign them busywork, but that doesn't take effort just as it wouldn't take effort for an adult to do the work.

In regular life we are all judged by others based on results, of course. When learning, however, you are best judged on effort. > Some kids are going to learn more quickly than others, and for them, no effort will be required. If no effort is required, then the bar is wrong.

As long as we don't have the resources to devote 10+% of the workforce to teaching, the bar will be wrong. The bar was wrong for me during school and university, and I found teachers who gave high weights to homework or even attendance quizzes to be extremely obnoxious.

On setting up expectations for adulhood, I think this is exactly backwards:

> If you got good grades at school with little actual effort and the constant praise for that formed your identity, you may be in for a bad time in adulthood.

Praising a child for effort without results seems way more likely to set them up for a surprisingly bad time as an adult. My personal experience has been that the "good grades/rewards without effort" thing has continued and seems pretty likely to continue through adulthood as long as you go into some kind of engineering.

Re: Do AI detectors work? Students face false cheating accusations

#504

Post by skhunted » Mon, Oct 21, 2024, 1:42 PM UTC

thechao wrote: post #378

abenga wrote: post #332

Earlier quoted context omitted.

Essays are async and easier to delegate.

If I'm paying 30k$/yr the professor is damn well reading my essay. If they don't want to teach & grade, they can get a pure research position. Fun fact: pure research positions don't pay as well.

Roughly 50% of higher education occurs at community colleges. We don’t do research. What you pay for the class does not correspond to what I make. I’m not paid enough to do all the stuff that is suggested in the comments.

The top earning professors in the nation in mathematics are all very good research mathematicians

Re: Do AI detectors work? Students face false cheating accusations

#505

Post by Jcampuzano2 » Mon, Oct 21, 2024, 1:43 PM UTC

skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

Agree. This isn't even necessarily an AI problem, people have been cheating/plagiarizing for years. And schools have failed to find or implement a method to prevent it.

I was in high school when kids started getting cell phones with internet access and basically as soon as that happened it opened up rampant cheating even among the best of students. I can only imagine it being much worse nowadays than even 15 years ago when I was in high school.

Re: Do AI detectors work? Students face false cheating accusations

#506

Post by lcnPylGDnU4H9OF » Mon, Oct 21, 2024, 1:44 PM UTC

iLoveOncall wrote: post #343

mrweasel wrote: post #184

The part that annoys me is that students apparently have no right to be told why the AI flagged their work. For any process where an computer is allowed to judge people, where should be a rule in place that demands that the algorithm be able explains EXACTLY why it flagged this person. Now this would effectively kill off the current AI powered solution, because they have no way of explaining, or even understanding, w…

Surely you understand how any algorithm (regardless of its nature) that gives the cheater the list of reasons why it spotted cheating will only work for a single iteration before the cheaters adapt, right?

I don’t think there’s anything to indicate they don’t understand this idea. But this misses the point; in their eyes, the lesser evil is to allow those with false positives to call the reasoning into question.

Re: Do AI detectors work? Students face false cheating accusations

#507

Post by max51 » Mon, Oct 21, 2024, 1:47 PM UTC

bdzr wrote: post #460

fuzzy_biscuit wrote: post #395

If AI detection cannot be 100% accurate, I do not believe it is an appropriate solution for judging the futures of millions of students and young people. Time to move on. Either from the tech or from the essay format. In either case, we need to change our standards around mastery of subject matter.

What solutions are 100% accurate?

The problem is that AI detection is far closer to 0% than 100%,. It's really bad and the very nature of this tech makes it impossible to be good.

Re: Do AI detectors work? Students face false cheating accusations

#508

Post by baby_souffle » Mon, Oct 21, 2024, 1:48 PM UTC

freilanzer wrote: post #462

raincole wrote: post #353

Earlier quoted context omitted.

Statistical models (which "AI" is) have been used to evaluate people's outputs since forever. Examples: Spam detection, copyrighted material detection, etc.

But not in cheating or grades, etc. Spam filters are completely different from this.

> But not in cheating or grades, etc. Spam filters are completely different from this.

Really? A spammer is trying to ace a test where my attention is the prize. I don't really see a huge difference between a student/diploma and a spammer/my attention.

Education tech companies have been playing with ML and similar tech that is "AI adjacent" for decades. If you went to school in the US any time after computers entered the class room, you probably had some exposure to a machine generated/scored test. That data was used to tailor lessons to pupil interest/goals/state curricula. Good software also gave instructor feedback about where each student/cohort is struggling or not.

LLMs are just an evolution of tech that's been pretty well integrated into academic life for a while now. Was anything in academia prepared for this evolution? No. But banning it outright isn't going to work

Re: Do AI detectors work? Students face false cheating accusations

#509

Post by vundercind » Mon, Oct 21, 2024, 1:50 PM UTC

pjc50 wrote: post #205

lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

> I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us. That basically never happens and the outcome is the result of some sort of struggle. Usually just a peaceful one in the courts and legislatures and markets, but a struggle nonetheless. > new methods should be used to assess student performance. Such as? We need an answer now because students are bei…

> Perhaps we're going to have to accept that and aggressively ration higher education by the limited amount of time available for human-to-human evaluations.

This will be it. [edit: for all education I mean, not just college] Computers are going to become a bigger part of education for the masses, for cost reasons, and elite education will continue to be performed pretty much entirely by humans.

We better hope computer learning systems get a lot better than they’ve been so far, because that’s the future for the masses in the expensive-labor developed world. Certainly in the US, anyway. Otherwise the gap in education quality between the haves and have nots is about to get even worse.

Public schools are already well on the way down that path, over the last few years, spurred by Covid and an increasingly-bad teacher shortage.

Re: Do AI detectors work? Students face false cheating accusations

#510

Post by sealeck » Mon, Oct 21, 2024, 1:51 PM UTC

skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

I think this is one of positives of standardised public exams (e.g. IB, Abitur, A Levels, etc); the people implementing them take cheating very seriously.