Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Professor's invisible prompt trap catches 32/35 students cheating with AI

techspot.com

Post reply on HN
Print view View original

41–50 of 99 posts

Re: Professor's invisible prompt trap catches 32/35 students cheating with AI

#41

Post by ColdStream »

vouaobrasil wrote: post #32
hatthew wrote: post #25

Every few weeks I see a headline about a teacher catching 80%+ of their students cheating with AI. I wonder how much newsworthiness selection bias is taking place here? Out of all teachers that do this type of experiment, do 90% of them them find that most students cheat, or do we just hear about the 1% that have the biggest numbers?

I've spent a lot of time in academia and the fact is, people will cheat if they can. That's why you have to implement proctoring and oral exams to prevent cheating. Because if cheating is a strategy that gets you a job and money, it will be implemented regardless. People are not ethical on average.

I think it is because we are implementing Goodharts law on a society scale.

"When a measure becomes a target, it ceases to be a good measure"

Many people see others taking short cuts and getting rewarded so that is seen as a legitimate path now. This works short term but errodes the foundations of that society over time.

So yeah this tracks for how things are going.

Re: Professor's invisible prompt trap catches 32/35 students cheating with AI

#42

Post by Cider9986 »

acmnrs wrote: post #12

I wonder if any of the three other students did use AI, but removed the Madagascar signal? So many students not proofreading suggests that they just want to make it look like they did the work but don't care about the grade, which is surprising for a midterm. It's also a little ironic that he's showing all of this while wearing Ray-Ban Metas.

I would get locked in if I noticed my professor using hidden text.

Re: Professor's invisible prompt trap catches 32/35 students cheating with AI

#44

Post by alcidesfonseca »

hatthew wrote: post #25

Every few weeks I see a headline about a teacher catching 80%+ of their students cheating with AI. I wonder how much newsworthiness selection bias is taking place here? Out of all teachers that do this type of experiment, do 90% of them them find that most students cheat, or do we just hear about the 1% that have the biggest numbers?

Professor here. Every single one of us that looks for it, finds plenty of examples of students “cheating”. Learning takes effort, and humans (some more than others) are designed to minimize effort.

We have designed education around exams and evaluation. If I design an exercise that does not contribute to their final grade, students don’t even look at it. If I design a 30min tutorial on a programming subject, students will not look at it and will spend 8 hours on the (graded) assignment that was designed for 1 hour on top of the 30 minute tutorial.

I am making some changes: evaluation will be as randomly as possible to minimize overfitting to our exercises. Draw from real world if possible.

I would also like to get rid of bologna’s modular structure and go to the UK’s one-exam-at-the-end-of-the-degree model.

But overall, we need more passionate students. Those don’t care about all this cheating and would appreciate that professors and future employers see their effort and success.

Re: Professor's invisible prompt trap catches 32/35 students cheating with AI

#46

Post by chucksmash »

hatthew wrote: post #25

Every few weeks I see a headline about a teacher catching 80%+ of their students cheating with AI. I wonder how much newsworthiness selection bias is taking place here? Out of all teachers that do this type of experiment, do 90% of them them find that most students cheat, or do we just hear about the 1% that have the biggest numbers?

It's endemic and it predates AI. I took a CS class 15 years ago where >40% of the class was caught copying the class project off of pastebin (no clue if it was specifically bait or student code that had a tell-tale error).

Re: Professor's invisible prompt trap catches 32/35 students cheating with AI

#47

Post by miki123211 »

If you do this sort of thing, for the love of God, test it with screen readers. And Reader mode. And probably a bunch of other things neither I nor you can think of, but that students will absolutely use, and then be falsely accused of cheating.

As a screen reader user, I have seen plenty of real world examples of people trying to be clever with white-on-white. The Joke is on them, I got their ID numbers and the secret instructions they gave to their coworkers. I even got my IP banned because of this once (from Arxiv I believe), back in the day when they had a "please don't click this" link on their website as an anti-crawler measure, which was accidentally made visible to screen readers.

Re: Professor's invisible prompt trap catches 32/35 students cheating with AI

#48

Post by epolanski »

cmiles8 wrote: post #4

I fear for this generation of folks that are losing the ability to think. They have a very sad existence ahead.

The ability to think has never been nurtured nor valued by the education system much, passing exams with high grades was.

That was always the goal and what the adult world would rate anyway.

And that meant that memory and repetition ended up being the core of the education system and it works very well from law to very advanced math where memorizing proofs and substitution rules is most of the curriculum anyway.

Problem solving, critical thinking, these things never really showed up or were valued by the system.

It was always the grades.

So a system that has emphasized the wrong metrics has always been gamified. AI is only speeding this gamification.

Re: Professor's invisible prompt trap catches 32/35 students cheating with AI

#49

Post by HPsquared »

ColdStream wrote: post #41
vouaobrasil wrote: post #32

Earlier quoted context omitted.

I've spent a lot of time in academia and the fact is, people will cheat if they can. That's why you have to implement proctoring and oral exams to prevent cheating. Because if cheating is a strategy that gets you a job and money, it will be implemented regardless. People are not ethical on average.

I think it is because we are implementing Goodharts law on a society scale. "When a measure becomes a target, it ceases to be a good measure" Many people see others taking short cuts and getting rewarded so that is seen as a legitimate path now. This works short term but errodes the foundations of that society over time. So yeah this tracks for how things are going.

Having a degree itself used to be a pretty good measure of something, and now it's pretty meaningless. (Unless you don't have one of course)

Re: Professor's invisible prompt trap catches 32/35 students cheating with AI

#50

Post by SunshineTheCat »

cmiles8 wrote: post #4

I fear for this generation of folks that are losing the ability to think. They have a very sad existence ahead.

I think there's some truth to this, however, I remember hearing similar arguments when people where using calculators on things they were suppose to be working out by hand.

I bet if you had one of these students explain the rationale behind a strategy they put together for winning at TFT, he or she would likely give you a very well thought out, coherent, and smart line of thinking.

If I had to guess, I would bet that it has far less to do with people being "unwilling to think" and far more of kids becoming exhausted by the boring, learn by memorization, pad your writing to fill space, monotony of an old and outdated education system, divorced from anything at all resembling the real world.

Speaking from experience, I did horrible in school and 100% would've been using ChatGPT whenever I could just to get through it.

The second I got out of school and found out work is far more result/merit based, I "learned" really, really quickly.

The old Mark Twain classic: “I have never let my schooling interfere with my education.”

Post reply on HN