Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

Post reply on HN
Print view View original

621–630 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#621

Post by zahlman »

teekert wrote: post #243

FWIW, I'm a consultant for a large University hospital, and Dutch. My PhD thesis, years ago, got the remark: "Should have checked with a native speaker." So, now I use ChatGPT to check my English. I just write what I want to write than ask it to make my text more "More concise, business-like and not so American" (yeah the thing is by default as ultra enthusiastic as an American waiter). And 9 out of 10 times it says…

>It's a Large Language Model, you should just is like that, it is not a Large Fact Model.

Not by design, but the training corpus necessarily includes a lot of "facts" (claims made by whoever wrote the original text). A model that is trying to output nonfiction on a specific topic, is likely to encounter relatively more models of claims that either actually were incidentally true, or at least have the same general form as true claims without an obvious "tell".

Of course, every now and then it goes off the rails and "hallucinates". Bad luck for the student who doesn't verify the output when this happens (which is probably a lot of students, since part of the motivation to cheat is not knowing the material well enough to do such verification properly).

Re: Do AI detectors work? Students face false cheating accusations

#622

Post by pdonis »

sevensor wrote: post #616
ericjmorey wrote: post #562

Earlier quoted context omitted.

I think you have a clearer understanding of the signalling that colleges have been providing for centuries than others who have been sold the lies that have been perpetuated by school administrators and those trying to justify their social advantages to those that didn't have similar advantages.

In a weird paradox, students who believe the lie and actually study to learn the material get more value from their education.

> get more value

If actually learning is valuable to them, independent of whether it will actually help them with prospective employers, then yes. But I don't think we can assume that all students value that.

Re: Do AI detectors work? Students face false cheating accusations

#623

Post by dataflow »

lumost wrote: post #611
skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

My personal take, we’ve made the cost of failure to high and cheating too easy. As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating. As someone who studied physics and came out with a 2.7 GPA due to studying what I wanted…

> As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating.

That's not the cost of not getting an A, it's the cost of appearing to underperform compared to too many of your peers. Which is directly tied to how many of them cheat. If not enough cheaters got an A then the cost would no longer be tied to not getting an A, it would be tied to whatever metric they appeared to outperform you on.

Re: Do AI detectors work? Students face false cheating accusations

#624

Post by _ea1k »

lumost wrote: post #611
skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

My personal take, we’ve made the cost of failure to high and cheating too easy. As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating. As someone who studied physics and came out with a 2.7 GPA due to studying what I wanted…

This really can't be emphasized enough. Universities and the initial hiring process really optimize for a score and not for learning. Those could be, and sometimes are, correlated, but it isn't necessarily the case.

Really focusing on stretching yourself necessarily means lower grades. Why is that penalized? TBH, in software engineering a lot of people with lower grades tutor the ones with 4.0 averages. The skillsets required to code and the skillsets required to get a good grade on a test are different.

Re: Do AI detectors work? Students face false cheating accusations

#625

Post by zamadatix »

gklitz wrote: post #330
lwhi wrote: post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

Written assay evaluation is not and has never been an effective evaluation. It was always a cost saving measure because allocating 30min face to face time with each individual student for each class is such a gigantic cost for the institution that they cannot even imagine doing it. Think about that the next time you look at your student debt, it couldn’t even buy you 30min time per class individually with the teacher…

> It was always a cost saving measure because allocating 30min face to face time with each individual student for each class is such a gigantic cost for the institution that they cannot even imagine doing it. Think about that the next time you look at your student debt, it couldn’t even buy you 30min time per class individually with the teacher to evaluate your performance.

Average student debt after a 4 year degree is ~$35,000 after ~45 courses. Before even running the math it should be obvious the gigantic cost of higher ed over 4 years is entirely unrelated to what an instructor would be making for ~23 hours of work (barring a secret society of multi millionaires). I.e. the problem you're identifying is the vast majority of $ spent in higher ed is not going to time with your professors, not that doing so is itself expensive.

Re: Do AI detectors work? Students face false cheating accusations

#626

Post by rangestransform »

retaJl wrote: post #207

I'm surprised at the number of comments that give up and say that "AI" is here to stay. I'm also surprised that academics rely on snake oil software to deal with the issue. Instead, academics should unite and push for outlawing "AI" or make it difficult to get like cigarettes. Sometimes politicians still listen to academics. It is probably not going to happen though since the level of political apathy among academics…

the cat is irreversibly out of the bag now, unless you want to ban gaming-grade GPUs, macbooks, and anything with high bandwidth memory capable of massively parallel compute. you can't strip the knowledge of how to build an LLM from people's brains, even non-ML software engineers will know the general research direction of how to get back to at least a GPT-3 level.

this is also not a good era for politicians to listen to academics, anti elitism sentiment is at a high and nobody will vote for "eating their vegetables" vs. "candy for dinner".

Re: Do AI detectors work? Students face false cheating accusations

#627

Post by smallnamespace »

account42 wrote: post #603
Jcampuzano2 wrote: post #574

Earlier quoted context omitted.

It seems more strange in my opinion that you'd never have a course thats popular enough that more than one teacher holds sessions for it. You don't have the choice to not take the class, you just have choice with which professor you would like to take it with. And often you would have to get lucky anyway, since that session may be filled so you'd have to take it with the "harder" teacher anyway. For example with the…

Why would a university need multiple professorts seaching the same subject at the same time? A professor isn't a school teacher that needs to look after each student individually. And even for questions andexcercises those are often already handled by teaching assistants of which there can be many as needed. Having the choice between different professors with supposedly different difficulties for what is supposed to…

> A professor isn't a school teacher that needs to look after each student individually

There's a line of research that shows that high quality one-on-one instruction gets you up to 2stdev gains in learning performance.

If you can afford to increase the professor to student ratio and make them available for office hours, you probably do see increases in performance. Is it due to better motivation? Seeing an academic up close? Actually better explanations you get from an expert in the subject? Hard to say.

Re: Do AI detectors work? Students face false cheating accusations

#628

Post by realitychx2020 »

VancouverMan wrote: post #495
red_admiral wrote: post #374

Earlier quoted context omitted.

If you're building a system to proctor students, then part of your job is to get it to work under all reasonable real-world conditions you might encounter: low light, students with standard webcams or just the one built into their laptop, students with darker skin etc. Reality might make this harder for some cases, but solving that is what you are being paid for. Also, this could have been handled much better in the…

The last time I got an ID photo taken, I got to wait and watch as the dark-skinned Indian photographer repeatedly struggled to take a suitable passport photo of the light-skinned white woman who was in line directly ahead of me. This was at a long-established mall shop that specialized in photography products and services. The same photographer had taken suitable photos of some other people in line ahead of us rather…

>> It has nothing to do with "racism".

Every major system in the US academic system is aimed to reducing Asian population. It often comes in the guise of DEI with a very wide definition of "Diversity" that rarely includes Asian.

These systems will use subtle features to blackbox racism. They may just be overt and leak over metadata to achieve it, or get smart and using writing styles.

Re: Do AI detectors work? Students face false cheating accusations

#630

Post by gs17 »

freilanzer wrote: post #462
raincole wrote: post #353

Earlier quoted context omitted.

Statistical models (which "AI" is) have been used to evaluate people's outputs since forever. Examples: Spam detection, copyrighted material detection, etc.

But not in cheating or grades, etc. Spam filters are completely different from this.

> But not in cheating or grades

I had both, over a decade ago in high school. Plagiarism detection is the original AI detection, although they usually told you specifically what you were accused of stealing from. A computer-based English course I took over the summer used automated grading to decide if what you wrote was good enough (IIRC they did have a human look over it at some point).

Post reply on HN