Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

Post reply on HN
Print view View original

371–380 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#371

Post by hnlmorg »

throwaway290 wrote: post #149
hnlmorg wrote: post #148

Earlier quoted context omitted.

Are you then plagiarising if the LLM is just regurgitating stuff you’d personally written? The point of these detectors is to spot stuff the students didn’t research and write themselves. But if the corpus is your own written material then you’ve already done the work yourself.

LLM is just regurgitating stuff as a principle. You can request someone else's style. People who are easy to detect simply don't do that. But they will learn quickly

I’ve found LLMs to be relatively poor at writing in someone else’s style beyond superficial / comical styles like “pirate” or “Shakespeare”.

To get an LLM to generate content in your own writing, there’s going to be no substitute for training it on your own corpus. By which point you might as well do the work yourself.

The whole point cheating is to avoid doing the work. Building your own corpus requires doing that work.

Re: Do AI detectors work? Students face false cheating accusations

#372

Post by _heimdall »

smartmic wrote: post #341
fullstackchris wrote: post #307

Earlier quoted context omitted.

I'm definitely no AI hypster, but saying anything will "never" work over an infinite timeline is a big statement... do you have grounds why some sort of AI system could one day "never" work at evaluating some metric about someone? Seems we have reliable systems already doing that in some areas (facial recognition at airport boarding, for example)

Okay, let me try to be more precise. By "evaluate", I mean using an AI to make predictions about human behavior , either retrospectively (as is the case here in trying to make an accusation of cheating) or prospectively (i.e. automating criminal justice). Even if you could collect all the parameters (features?) that make up a human being, there is the randomness in humans and in nature in general, which simply destro…

This feels like an argument bigger than AI evaluations. All points you raised could very well be issues with humans evaluating other humans to attempt to predict future outcomes.

Re: Do AI detectors work? Students face false cheating accusations

#373

Post by johnisgood »

_heimdall wrote: post #360
johnisgood wrote: post #325

Earlier quoted context omitted.

Physically laborious work is an increasing problem as you age though.

Historically that's more a question about community. Its a very recent phenomenon to have cultures where parents and grandparents are expected to take care of themselves or live in a home/facility.

Living in an elderly home may be impossible, too[1], meaning at best you can stay at the hospital until you die (which doctors are eager to achieve), at least in Hungary.

There is a sad, depressing world out there. One of my parents work at an elderly home, and the shit that happens there is just wild. Zero responsibility and accountability. Deliberate killing of people out of pure inconvenience, etc.

I am in favor of a "social support network".

[1] Requires money, e.g. pension, which is increasingly less, and they keep increasing the age.

Re: Do AI detectors work? Students face false cheating accusations

#374

Post by red_admiral »

15155 wrote: post #322
Iulioh wrote: post #296

Earlier quoted context omitted.

The "dark skin problem" is mostly the camera sensors, not only the training... Low light scenarios are just a thing, you would need more expensive hardware do deal with it.

> mostly the camera sensors Could it be mostly just be..reality? More expensive hardware doesn't somehow make a darker surface reflect more energy in the visible spectrum. "Low light" is not the same condition as "dark surface in well-lit environment." Leaving the visible spectrum is one possible solution, but it's substantially more error-prone and costly. This is still not the same solution as classical CV with "mo…

If you're building a system to proctor students, then part of your job is to get it to work under all reasonable real-world conditions you might encounter: low light, students with standard webcams or just the one built into their laptop, students with darker skin etc. Reality might make this harder for some cases, but solving that is what you are being paid for.

Also, this could have been handled much better in the cases that came up in the media if there had been proper human review of all cases before prosecuting the students.

Re: Do AI detectors work? Students face false cheating accusations

#375

Post by ransom1538 »

lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

IMHO. With calculators introduced, there is zero add in you learning long division. Worse than zero, you could have done something better with your time. ChatGPT is a calculator for all subjects. People have a hard time letting that sink in.

Re: Do AI detectors work? Students face false cheating accusations

#376

Post by lukan »

ordu wrote: post #358
gklitz wrote: post #330

Earlier quoted context omitted.

Written assay evaluation is not and has never been an effective evaluation. It was always a cost saving measure because allocating 30min face to face time with each individual student for each class is such a gigantic cost for the institution that they cannot even imagine doing it. Think about that the next time you look at your student debt, it couldn’t even buy you 30min time per class individually with the teacher…

> It was always a cost saving measure because allocating 30min face to face time with each individual student for each class is such a gigantic cost for the institution that they cannot even imagine doing it. So the obvious solution is to make students to talk with an AI, which would grade their performance. Or, maybe the grading itself could be done by a minimum wage assistant, while AI would lead the discussion wit…

I hope that was sarcasm?

Re: Do AI detectors work? Students face false cheating accusations

#377

Post by ninalanyon »

michaelt wrote: post #254
pjc50 wrote: post #205

Earlier quoted context omitted.

> I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us. That basically never happens and the outcome is the result of some sort of struggle. Usually just a peaceful one in the courts and legislatures and markets, but a struggle nonetheless. > new methods should be used to assess student performance. Such as? We need an answer now because students are bei…

> Such as? We need an answer now because students are being assessed now. Two decades ago, when I was in engineering school, grades were 90% based on in-person, proctored, handwritten exams. So assignments had enough weight to be worth completing, but little enough that if someone cheated, it didn't really matter as the exam was the deciding factor. > Return to the old "viva voce" exam? Still used for PhDs. But that…

There are institutions that still require a public defence for a PhD, not merely a viva. Oslo University for instance: https://www.uio.no/english/research/phd/

Re: Do AI detectors work? Students face false cheating accusations

#378

Post by thechao »

abenga wrote: post #332
kubb wrote: post #300

Earlier quoted context omitted.

But you can read 200 essays? At this point you can be replaced with AI, you’re not adding any value anymore.

Essays are async and easier to delegate.

If I'm paying 30k$/yr the professor is damn well reading my essay. If they don't want to teach & grade, they can get a pure research position. Fun fact: pure research positions don't pay as well.

Re: Do AI detectors work? Students face false cheating accusations

#379

Post by hyperbrainer »

arkh wrote: post #271
hyperbrainer wrote: post #140

Earlier quoted context omitted.

At least the first 2 are far more accurate than humans ever could be. The third, i.e. trusting others to vet and find the correct information, is the problem.

> trusting others to vet and find the correct information, is the problem To be honest, we do for most things: I have not checked the speed of light. And I surely would not be able to implement a way to measure it from only my observations and experience.

Agreed, but google hardly gives you those results. Sponsored Ads and AI generated seo crap is hardly an encylopedia

Re: Do AI detectors work? Students face false cheating accusations

#380

Post by hyperbrainer »

fullstackchris wrote: post #318
ClumsyPilot wrote: post #209

Earlier quoted context omitted.

I just googled ‘do I need a license to drive a power boat in UK’ I got AI answer saying ‘no’, but actually you do. If I use a calculator it will be correct. If I open encyclopaedia it will mostly be correct, because someone with a brain did at least 5 minutes of thining. We are not talking about some minor detail, AI makes colossal errors with great confidence and conviction.

But you're comparing apples to oranges anyway... a mathematical problem is vastly different than a q&a problem - which of course involves language which is anyway a lossy form of communication.

that is the point. google is not Multivac
Post reply on HN