Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

Post reply on HN
Print view View original

711–720 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#711

Post by Syonyk »

QuercusMax wrote: post #705
vkou wrote: post #695

Earlier quoted context omitted.

> On-location tests/leet code are a crapshoot They aren't 'fair' in avoiding every false negative, but they at least tell me that the passing candidates know and can do something. If I ask someone who claims to know Python or Java whether or not you can have a memory leak in them, and their answer is 'no' or 'Maybe, but I don't know how', I get a pretty good idea of whether or not they know anything about this topic.…

Back when I was at a small company doing a lot of new-grad interviews, it was really shocking how many people couldn't solve fizzbuzz or something equally trivial, like reversing an array in-place. When I was at Google, most of these were filtered out before I got to see them, but for a while I was doing iOS development interviews and a lot of the candidates applying to Google clearly didn't know anything .

The first time I saw FizzBuzz, I immediately assumed it was some sort of "trap" or "trick" interview question - that there's some deviously subtle little thing in it that you'll miss at first or second glance, as a "gotcha." It literally never occurred to me that it was, in fact, a basic "Can you code your way out of a paper bag given a map?" sort of question to check for basic code competence in languages.

Then I started interviewing, and... yeah. I get it now. It really is that simple, should take a competent coder a few minutes, and 80% of people interviewing will take 45 minutes to muddle their way through it.

Re: Do AI detectors work? Students face false cheating accusations

#712

Post by lcnPylGDnU4H9OF »

tipsytoad wrote: post #297

I don’t think this hits at the heart of the issue? Even if we can catch AI text with 100% accuracy, any halfway decent student can rewrite it from scratch using o1s ideas in lieu of actually learning. This is waay more common and just impossible to catch. The only students caught here are those that put no effort in at all

> rewrite it from scratch ... in lieu of actual learning

If one can "rewrite it from scratch" in a way that's actually coherent and gets facts correct, then they learned the material and can write an original paper.

> This is waay more common and just impossible to catch.

It seems a good thing that this is more common and, naturally, it would -- perhaps should, given the topic -- be impossible to catch someone cheating when they're not cheating.

Re: Do AI detectors work? Students face false cheating accusations

#713

Post by amanaplanacanal »

fallingknife wrote: post #581
skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

They dumbed down college degrees so that everyone can get one. What did you expect? Can't do that without lowering standards.

We are now several generations in on telling people the way to get a good job is to get a college degree. So everybody is there to get the piece of paper, not to actually learn things they are interested in.

Re: Do AI detectors work? Students face false cheating accusations

#715

Post by hedvig23 »

Suppafly wrote: post #677
nostrademons wrote: post #541

Earlier quoted context omitted.

The solution, clearly, is a world where those who actually learned the math can use it to cheat the people who didn't. ...which is what we have today, where the most lucrative industries for people with good math skills are finance (= cheating dumb people out of their retirement), advertising (= cheating dumb people out of their consumer dollars), and data-driven propaganda (= cheating dumb people out of their votes)…

The people who actually learned the math work in STEM careers, not fancied up sales careers.

Yes and if STEM industry is Silicon Valley then that is just advertising ultimately or if not ads, something much more immoral, data collection for social control. Which is advertising's intention as well so I guess all the same work

Re: Do AI detectors work? Students face false cheating accusations

#716

Post by busyant »

skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

> The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test.

I completely agree, but the entire higher ed system is moving to on-line instruction.

Basically, if the University of follows your suggestion, all of the competing institutions will eat their lunch by offering on-line courses with the "convenience" of on-line assessments" and the University of will lose enrollment.

:-(

Re: Do AI detectors work? Students face false cheating accusations

#717

Post by fsndz »

lumost wrote: post #611
skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

My personal take, we’ve made the cost of failure to high and cheating too easy. As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating. As someone who studied physics and came out with a 2.7 GPA due to studying what I wanted…

> My personal take, we’ve made the cost of failure to high and cheating too easy. This is so true. I was recently pondering about the impact of AI cheating in Africa and came up with the conclusion that it won't be as significant as in EU/US precisely because most evaluations in African countries are in person https://www.lycee.ai/blog/can-africa-leapfrog-its-way-to-ai-... Your take reminds me of Goodhart's law: "When a measure becomes a target, it ceases to be a good measure". Same is true with GPA and all. But I am pessimistic about seing that change in the medium to long term because it is so politically sensitive.

Re: Do AI detectors work? Students face false cheating accusations

#718

Post by zero-sharp »

skhunted wrote: post #596
_fat_santa wrote: post #576

Earlier quoted context omitted.

I remember taking a math class in college and the professor had a very unique way of dealing with cheating. He let us use our books, notes, and "any calculator capability" from our TI-84's. His rationale is that students will try to use these tricks anyways so just let them and then update the test to be "immune" from these advantages. Before every test he mentioned that we could use all those tools but always said "…

I’ve done this but then you end up with students who are not used to “thinking”. They do bad on the test. Now I’m known as a hard teacher. Now people avoid my classes. Administration hounds me for having s low passing rate. I need a job. I now give easy tests. The real issue as I see it is that no one wants to face the reality that far too many incapable, incurious people are going to college. So I pretend to give re…

I was involved in K-12 math education for a few years and there's absolutely a pressure to make things easy for kids. When certain parents see Johnny scored poorly on a test, guess what they do? They start a conversation with the teacher and administration. Johnny needs to pass, or maybe even succeed, and it's the education that has to change around him. It creates more work. Teaching already isn't a traditional 9-5. Grading homework can consume hours outside of normal working time. Meanwhile, I can count on one hand how many times I needed to put in overtime at my office job.

If the school has a tuition, then there's even more of a conflict of interest. I've had parents/admins imply that we might be losing a student due to poor grades.

Re: Do AI detectors work? Students face false cheating accusations

#719

Post by rahimnathwani »

greatartiste wrote: post #111

For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy. Text seems to use the same general framework (although words are swapped around) also we see what I call 'word of the week' where whichever 'AI' engine seems to get hung up on a particular English word which is often an unusual one and uses it at every opportunity. It isn't long before you real…

  For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy.
When evaluating job applications we don't have ground truth labels, so we cannot possibly know the precision or recall of our classification.

Re: Do AI detectors work? Students face false cheating accusations

#720

Post by golergka »

skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

I never understood why americans do their exams with multi-option tests. Even if you don't cheat, these tests don't actually test knowledge, just memoization.

For me a proper exam is when you get a topic, spend 30 minutes in a classroom preparing, and then sit down with an examiner to tell him about this topic and answer all the follow-up questions.

We don't do multi-option tests at software interviews, and for a good reason. Why do them in a uni?

Post reply on HN