Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

Post reply on HN
Print view View original

811–820 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#811

Post by floatrock »

mrweasel wrote: post #184

The part that annoys me is that students apparently have no right to be told why the AI flagged their work. For any process where an computer is allowed to judge people, where should be a rule in place that demands that the algorithm be able explains EXACTLY why it flagged this person. Now this would effectively kill off the current AI powered solution, because they have no way of explaining, or even understanding, w…

Must be so demoralizing to be a kid these days. You use AI --> you're told you're cheating, which is immoral. You don't use AI --> you eventually get accused of using it or you get left behind by those who do use it.

Figuring out who the hell you are in your high school years was hard enough when Kafka was only a reading assignment.

Re: Do AI detectors work? Students face false cheating accusations

#812

Post by itishappy »

BobaFloutist wrote: post #782
itishappy wrote: post #657

Earlier quoted context omitted.

> For an English writing assignment, that's cheating. Whoops, with that little comment I suspect you've invalidated most English papers written in the past 2 decades. Certainly all of mine! Thanks spellcheck.

Grammarly is very different from vanilla spellcheck.

Fair enough. My last exposure to Grammarly was pre-ChatGPT, when it was a lot closer to vanilla spellcheck.

But I think it's actually not all that different, particularly in the context of "essays teach writing." It used to be human work to analyze sentences for passive voice, remember the difference between there/their/they're, and understand how commas work, but now the computer handles it.

(Relevant sidenote: Am I using commas correctly here? IDK! I've never fully internalized the rules!)

Re: Do AI detectors work? Students face false cheating accusations

#813

Post by SkyBelow »

Izkata wrote: post #534
obscurette wrote: post #408

Earlier quoted context omitted.

It's actually not about beauty of the math, it's about something which is nowadays called a number sense. It takes a lot of practice to develop an understanding what these things called numbers are, how these relate to each other, what happens if you combine these with operational signs, how numbers grow and shrink etc. And you are damn right that there is no any use to explain it to the 12 year olds. Or even to 16 y…

Common Core was an attempt at teaching this directly. It gets so much ridicule because so few people have good enough number sense to recognize what they're seeing when shown a demonstration. Of course, since they didn't understand it, it then led to bad examples being created and shared, which just made it worse...

It was more than that.

It didn't explain the goals well enough to parents, and many teachers didn't have the number sense themselves leading to many of the examples are passed around showing how the whole process is broken. There is also a question of if even works well, as it is somewhat akin to teaching someone the shortcut on how to do something before they have mastered the long way of doing it. Many experts in their fields have shortcuts, but they don't teach them directly to juniors in the field as there is value in learning how to do it the long hard way, as often times shortcuts are limited and only an understanding of the full process provides the knowledge of when best to apply different shortcuts.

Re: Do AI detectors work? Students face false cheating accusations

#814

Post by FloorEgg »

skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

I have friends that started a startup trying to tackle this problem. They actually found ways for certain types of exams in certain subjects to make cheating exponentially harder and also provide less of an advantage, so much so that if the student is cheating they are effectively learning.

Some of their stuff works really well, and they have prof customers who love it. The CEO went on a tour to visit their biggest customers in person and several of them said they couldn't imagine going back.

Unfortunately as a whole the industry is not interested in it, aside from a few small niches and department heads who are both open minded and actually care about the integrity of the education. There have even been cases where profs want it and the dean or admin in charge of academic integrity vetoes its adoption. I've been privy to some calls I can only characterize as corrupt.

There is something deeply broken about higher Ed, the economics, the culture of the students, the culture of the faculty, the leadership... This isn't an AI problem it's a society problem.

When the students genuinely want to learn something and they are there for the knowledge, not the credit, cheating isn't a problem.

Re: Do AI detectors work? Students face false cheating accusations

#815

Post by godelski »

_ea1k wrote: post #624
lumost wrote: post #611

Earlier quoted context omitted.

My personal take, we’ve made the cost of failure to high and cheating too easy. As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating. As someone who studied physics and came out with a 2.7 GPA due to studying what I wanted…

This really can't be emphasized enough. Universities and the initial hiring process really optimize for a score and not for learning. Those could be, and sometimes are, correlated, but it isn't necessarily the case. Really focusing on stretching yourself necessarily means lower grades. Why is that penalized? TBH, in software engineering a lot of people with lower grades tutor the ones with 4.0 averages. The skillsets…

  > Really focusing on stretching yourself necessarily means lower grades.
I'm reminded of a saying/trope (whatever) I've seen in reference to surgeons and lawyers (I'm sure it's also been used in TV and movies). But the trope is that someone is looking for an expert and will be talking to a bunch of hotshots (let's say lawyers). They'll be bragging and then asked if they've ever lost a case, to which they proudly declare they have a spotless record. To which the person responds: then you've never taken a single risk.

It's overly dramatic, but I think gets the point across in an easy to understand way. It's exactly why you see the lower grade ones tutor the high grade ones (this even happened in my undergrad and I did physics[0]).

It's because learning happens when struggling. It happens at the edge. This is also a big reason some learn a lot faster than others or even why someone will say they don't understand but understand more than someone who says they do (and who believes it). Because expertise isn't about the high level general ideas, it's about all the little nitty gritty details, the subtle things that dramatically change things. But a big concern I have is that this is a skill to learn in of itself. I think it's not difficult to recognize when this skill is learned (at least if you have) but it's not something that'll be learned if we focus to much on scores. After all, they're just a proxy. Even the institutional prestige is a proxy (and I have an argument why it no longer matters though it did decades ago).

I do wonder if this is in part cause for the rise in enshitification. Similarly if this is why so many are bad at recognizing issues in LLMs and ML models. I'm sure it is but not sure how much this contributes or if it's purely a confounding variable.

[0] when I signed up to be a tutor at my university I got signed off my the toughest math professor. When I took the signature to the department the admin wasn't sure if I was trying to trick her because she immediately called the professor to confirm the signature. Then told me I could tutor whatever I wanted because I was one of two people he had ever signed off on. Admittedly, I'm sure a lot of that was because people were afraid of him (he wasn't mean, but he wouldn't let you be anything less than the best he thought you could be)

Re: Do AI detectors work? Students face false cheating accusations

#816

Post by ninalanyon »

dayvid wrote: post #592
noodlesUK wrote: post #570

Earlier quoted context omitted.

Fellow UK person - the style of exam that you describe is pretty hard to cheat unless you can find another person to go in your place. I think various institutions have tried digital invigilation but have had little success (and I think this is just a bad idea anyway). However, you also mentioned a final project. You’d be shocked how much commissioning exists where people have their projects produced for them. I’m no…

Had a good friend who tutored college students and a rich middle-eastern student paid him to do a lot of his work for him.

That won't work in a tutorial system, the student will be quickly discovered to know nothing about the subject. And in open note finals, as in the Exeter Uni. Physics department of the 1970s, regurgitation of course material was of very limited utility because you were never asked for that kind of response. The quantum mechanics final didn't ask a single question that had been directly answered during lectures, it asked us to extend what we had learnt. That exam was what I think Americans might call a 'white knuckle ride'. Open note finals really sort those who understood the subject from those who thought they could just look up the answers, the invigilators spent a lot of time shushing people searching through rucksacks full of notes.

Many years later I took a course in C# at a university in Norway and that was not merely open note but also open book (you could take the set book in). Again that gives the exam author the possibility to really discover who knows what.

I doubt that your rich middle-western student would have passed either of these

Re: Do AI detectors work? Students face false cheating accusations

#817

Post by IncreasePosts »

skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

> Students are not used to doing the necessary work to learn. They are used to doing the necessary work to pass.

Can you blame them? If they do the necessary work to learn, but do poorly on an exam for some reason, will you still give them a passing grade?

Re: Do AI detectors work? Students face false cheating accusations

#818

Post by joe_the_user »

junaru wrote: post #351
lm28469 wrote: post #339

Earlier quoted context omitted.

In France my essays were written in class, no phones, no book, just your brain, a sheet a paper and a pen. That's still 100% doable today

It even came with handwriting built in as authentication mechanism! AI detectors hate this secret! On a more serious note - US removed cursive from their curriculum almost two decades ago - something i cant wrap my head around as cursive is something the rest of the world(?) uses starting in middle school and onwards through the whole adult life.

There's no particular reason people need to use cursive rather than printing. I personally always struggled with cursive due to eye-hand coordination issues and teachers' demand for it just seemed like hazing (and I'm a boomer). Good riddance to cursive.

Re: Do AI detectors work? Students face false cheating accusations

#819

Post by viraptor »

mrweasel wrote: post #340
viraptor wrote: post #282

Earlier quoted context omitted.

> kill off the current AI powered solution, because they have no way of explaining That's not correct. Some solution look at perplexity for specific models, some will look at ngram frequencies, and similar approaches. Almost all of those can produce a heatmap of "what looks suspicious". I wouldn't expect any of the detection systems to be like black boxes relying on LLM over the whole text.

Sorry if this is "moving the goal post", but I wouldn't call looking at ngram frequencies for AI. Producing a heatmap doesn't tell you why something is suspicious, but it's obviously better than telling you nothing. In any case, if you where to use LLMs, or other black box solutions, you'd have to yank those out, if you where met with a requirement to explain why something is suspicious.

It's literally the explanation. The only identification we have now is "this local part is often used by an AI model" and "this global structure is often used by an AI model". There's nothing more fancy about it. The heatmap would literally just point out "this part is suspiciously unlikely" - that's the explanation because that's the classification systems use.

Re: Do AI detectors work? Students face false cheating accusations

#820

Post by pnutjam »

michaelt wrote: post #651
Jcampuzano2 wrote: post #574

Earlier quoted context omitted.

It seems more strange in my opinion that you'd never have a course thats popular enough that more than one teacher holds sessions for it. You don't have the choice to not take the class, you just have choice with which professor you would like to take it with. And often you would have to get lucky anyway, since that session may be filled so you'd have to take it with the "harder" teacher anyway. For example with the…

> It seems more strange in my opinion that you'd never have a course thats popular enough that more than one teacher holds sessions for it. Remember, in European countries students are admitted to study a specific subject at university, rather than being admitted to the university as a whole and expected to choose a major later on. So there are multiple courses going on, with a lot of intersection between the topics…

> Remember, in European countries students are admitted to study a specific subject at university, rather than being admitted to the university as a whole and expected to choose a major later on.

This is true in the US as well. You can change your major, but you are admitted into a College in the University. Moving to another College is not guaranteed if you later change your mind.

Post reply on HN