Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

Post reply on HN
Print view View original

271–280 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#271

Post by arkh »

hyperbrainer wrote: post #140
arkh wrote: post #114

Earlier quoted context omitted.

We don't learn directions now: we use GPS. We don't do calculations: computers do it for us. We don't accumulate knowledge: we trust Google to give us the information when needed. Everything in a small package everyone can wear all day long. We're at the second step of transhumanism.

At least the first 2 are far more accurate than humans ever could be. The third, i.e. trusting others to vet and find the correct information, is the problem.

> trusting others to vet and find the correct information, is the problem

To be honest, we do for most things: I have not checked the speed of light. And I surely would not be able to implement a way to measure it from only my observations and experience.

Re: Do AI detectors work? Students face false cheating accusations

#272

Post by pca006132 »

Ukv wrote: post #257
pjc50 wrote: post #205

Earlier quoted context omitted.

> I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us. That basically never happens and the outcome is the result of some sort of struggle. Usually just a peaceful one in the courts and legislatures and markets, but a struggle nonetheless. > new methods should be used to assess student performance. Such as? We need an answer now because students are bei…

> Such as? We need an answer now because students are being assessed now . Return to the old "viva voce" exam? Still used for PhDs. But that doesn't scale at all. For a solution "now" to the cheating problem, regular exam conditions (on-site or remote proctoring) should still work more or less the same as they always have. I'd claim that the methods affected by LLMs are those that could already be circumvented by tho…

Yeah, LLM is kind of just making expensive cheats cheaper. You can do it without LLM, and indeed students did similar things prior to the release of ChatGPT, just less common.

Re: Do AI detectors work? Students face false cheating accusations

#273

Post by arkh »

ClumsyPilot wrote: post #230
arkh wrote: post #123

Earlier quoted context omitted.

It is not a bug, it is a feature. That's how you can mold society as you like at your level: this student's older sibling was a menace? Let's fuck them over, being shitty must run in the family. You don't like the race / gender / sexuality of a student? Now "chatGPT" can give you an easy way to make their school life harder.

This is not about ChatGPT. The same happens in HR departments And governments. Just introduce an incomprehensible process, Like applying for a Visa or planning permission, and then use it to your advantage. From the victims perspective, there is no difference between bureaucracy and AI

> This is not about ChatGPT.

I agree. But now some people can point to ChatGPT or other tools and use it as an excuse. So for them, the "bugs" are a feature. They don't care about false positives, they care about the fact some authority tells them a student they don't like used AI to write an essay.

Re: Do AI detectors work? Students face false cheating accusations

#274

Post by torginus »

lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

Just because a method of assessment became easily spoofable doesn't mean we should give up on it. Imagine if in the era before HTTPS we just said that the internet won't be really viable because it's impossible to communicate securely on it.

I still feel like AI detectors would work well if we have access to the exact model, output probabilities of tokens, We can just take a bit of given text, and calculate the cumulative probability that the AI would complete it exactly like that.

Re: Do AI detectors work? Students face false cheating accusations

#276

Post by logicchains »

pjc50 wrote: post #205
lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

> I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us. That basically never happens and the outcome is the result of some sort of struggle. Usually just a peaceful one in the courts and legislatures and markets, but a struggle nonetheless. > new methods should be used to assess student performance. Such as? We need an answer now because students are bei…

>Return to the old "viva voce" exam? Still used for PhDs. But that doesn't scale at all.

On the contrary; with AI it scales better than ever before.

Re: Do AI detectors work? Students face false cheating accusations

#277

Post by light_hue_1 »

michaelt wrote: post #254
pjc50 wrote: post #205

Earlier quoted context omitted.

> I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us. That basically never happens and the outcome is the result of some sort of struggle. Usually just a peaceful one in the courts and legislatures and markets, but a struggle nonetheless. > new methods should be used to assess student performance. Such as? We need an answer now because students are bei…

> Such as? We need an answer now because students are being assessed now. Two decades ago, when I was in engineering school, grades were 90% based on in-person, proctored, handwritten exams. So assignments had enough weight to be worth completing, but little enough that if someone cheated, it didn't really matter as the exam was the deciding factor. > Return to the old "viva voce" exam? Still used for PhDs. But that…

Oh yes. When I'm teaching a class of 200 students it's totally plausible that we're going to do 10 15 minute one on one conversations with every student. Because that's only 20 days non stop with no sleep.

We would need to increase the amount of teaching staff by well over 10x to do this. The costs would be astronomical.

Re: Do AI detectors work? Students face false cheating accusations

#278

Post by red_admiral »

lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

> It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject.

It never was. It's just even more ineffective now that AI exists, than before.

The central example of this is college admissions statements. Some kids have the advantage both of parents who can afford to give them the experiences that look good on such an essay (educational trips to Africa, lessons in two musical instruments, one-on-one golf coaching, that kind of thing), and who can hire tutors to "support" them in writing the essay. AI just makes the tutor part accessible/affordable for a wider segment of the population.

It would be naive to assume that, pre-AI, there was not a "gray" essay-coaching market as well as the "dark" essay-writing as a service market. That market still works better than AI in many cases.

Re: Do AI detectors work? Students face false cheating accusations

#279

Post by tomjen3 »

pjc50 wrote: post #205
lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

> I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us. That basically never happens and the outcome is the result of some sort of struggle. Usually just a peaceful one in the courts and legislatures and markets, but a struggle nonetheless. > new methods should be used to assess student performance. Such as? We need an answer now because students are bei…

Simple: you still write an essay and you may use ai to do so. Then you throw the essay out and go and talk with the teacher about it. If you can answer intelligently it’s because you know the stuff and if not then you don’t.

Re: Do AI detectors work? Students face false cheating accusations

#280

Post by tippytippytango »

prepend wrote: post #22

My kids’ school added a new weapons scanner as kids walk in the door. It’s powered by “AI.” They trust the AI quite a bit. However, the AI identifies the school issued Lenovo laptops as weapons. So every kid was flagged. Rather than stopping using such a stupid tool, they just have the kids remove their laptops before going through the scanner. I expect not smart enough people are buying “AI” products and trusting th…

This is what we really need AI regulation for. The accuracy rates should be advertised in a standard format like a nutritional label. People purchasing the systems on public dollars should be required to define a good plan for false positives and negatives that handles the expected rates based on the advertised precision and recall.
Post reply on HN