Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

Post reply on HN
Print view View original

821–830 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#821

Post by hungariantoast »

skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

> The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test.

If all my math professors had done this, I never would have earned my computer science degree or my minor in mathematics.

I have an immensely difficult time memorizing formulas and doing math by hand. I absolutely need to be able to prepare notes ahead of time, and reference them, to be able to complete a math test on paper. Even then, I'm a very slow in-person test-taker, and would often run out of time. I've honestly come around to the idea that maybe I have some sort of learning disability, but I never gave that idea much thought in college. So, I didn't qualify for extra time, or any other test-taking accommodations. I was just out-of-luck when time was up on a test.

The only reason I was able to earn my degree is because I was able to take almost all of my math classes online, and reference my notes during tests. (COVID was actually a huge help for this.)

And by "notes", I don't just mean formulas or solutions to example problems that I had recorded. I also mean any of the dozens of algorithms I programmed to help automate complex parts of larger problems.

The vast majority of the math classes I took, regardless of whether they were online or in-person, did not use multiple-choice answers, and we always had to show our work for credit. So I couldn't just "automate all the things!", or use AI. I did actually have to learn it and demonstrate how to solve the problems. My issue was that I struggled to learn the material the way the university demanded, or in their timeframe.

So as an otherwise successful student and capable programmer, who would have struggled immensely and been negatively affected mentally, professionally, and financially, had they been forced to work through math courses the way you prescribe, I'm asking you: please reconsider.

Please reconsider how important memorization should be to pass a math class, how strongly you might equate "memorized" to "learned", and what critical thinking and problem-solving could look like in a world where technology is encouraged as part of learning, not shunned.

Re: Do AI detectors work? Students face false cheating accusations

#822

Post by godelski »

withinboredom wrote: post #808
godelski wrote: post #752

Earlier quoted context omitted.

It's subtle, but people who are self driven and learning for the sake of learning will talk differently. They tend to include more nuance and detail, addressing the subtle things. To be able to see those things requires internalization of what's learned, not just memorization. If you get good at it, you can do pretty well at recognizing these people even when they're in a different subject domain. Remember, outside C…

> I'm also very sympathetic to the jitters stuff, because I get it a lot and make dumb mistakes when in the spotlight lol. As an interviewer, I spot this and try to get them to ease up. I will talk about myself for a bit, about the work I do. I'm trying to get them to realize they are not in the spotlight, but whether we would be a good fit together; and thus both of us want them to work there. BUT, my interviews ten…

I think what I'm saying is more important to the type of interviews you do. And I think for the most part we agree (or I misunderstand?). Those interviews sound much closer to the classic engineering interview (as in not programming but like mechanical or civil engineering) or typical science interview. I think those are better interviews and more meaningful than live coding sessions or whiteboard problems.

Maybe here's a general question you can add (if you don't already use it) to bring out that thinking even if they're nervous. Since it's systems they are familiar with (my forum entry example is similar. I don't do front end), ask them what things they're frustrated with in tools they've used and how they could be fixed. It can help to ask if they've tried different solutions. With email that can be like if they just use Gmail via the Web, just use outlook or Apple Mail, or have tried things like Thunderbird, mux, or other aggregators. Why do they like the one they use? And if they've tried others I think that in itself is a signal that they will look for improvements on their own.

The things I think many interviews do poorly at is that they tend to look for knowledge. I get this, it's the easiest thing to measure because it's tangible. It's something you "have". While this matters, the job is often more dependent on intelligence and wisdom which are more about inference, attention, flexibility, and extrapolation. So I don't think it's so much about "gotchas" -- especially as many now just measure how "prepared" they are -- but, like you said, the way they think.

I'd much rather take someone with less knowledge (within reason) who is more intelligent, curious, and/or self driven by the work (not external things like money or prestige). Especially with juniors. A junior is an investment and thus more about their potential. As they say, you cannot teach someone who "already knows".

[EDIT]:

There's something else I should bring up about the "classic engineering" interview. Often they will discuss a problem they are actively working on. A reason for this is 1) it is fresh in their mind, 2) it gets at details, *but* 3) because it makes it easier for the interviewee to say "I don't know."

I think this is often an issue and sometimes why people will say weird erroneous things. They feel pressured to not admit they don't know and under those conditions, a guess is probably a better strategy. Since admitting lack of knowledge is an automatic "failure" while a guess has some chance, even if very small. At least some will admit to guessing before they do and you can also say its fine to guess and I see that often relax people and frequently results in them not guessing and instead reason through it (usually out loud).

(I'm an older grad student finishing up, so I frequently am dealing with undergrads where I'm teaching a class, holding office hours, or mentoring them in the lab. I've done interviews when I was a full time employee before grad school, and I notice there's a lot of similarities in these situations. That people are afraid to admit lack of knowledge when there is an "expert" in front of them. Even if they are explicitly there to get knowledge from said expert.)

Re: Do AI detectors work? Students face false cheating accusations

#823

Post by Akranazon »

zahlman wrote: post #602
Akranazon wrote: post #567

Earlier quoted context omitted.

Germany isn't special, (almost) all exams work like that in the US as well. I don't know why he was implying otherwise. Almost all degrees have required courses in the US as well. You point to a true failure in incentives. And yet, the US has the highest density of renowned universities.

>And yet, the US has the highest density of renowned universities. The renown has to do with a lot more than demonstrated ability of graduates.

[dead]

Re: Do AI detectors work? Students face false cheating accusations

#824

Post by physicsguy »

ninalanyon wrote: post #365
gklitz wrote: post #330

Earlier quoted context omitted.

Written assay evaluation is not and has never been an effective evaluation. It was always a cost saving measure because allocating 30min face to face time with each individual student for each class is such a gigantic cost for the institution that they cannot even imagine doing it. Think about that the next time you look at your student debt, it couldn’t even buy you 30min time per class individually with the teacher…

When I studied physics at Exeter University they still used the tutorial system and finals. Tutorials were held fortnightly; the tutorial groups were typically three or four students. There was no obligation to turn up to lectures or even tutorials. You just had to pass the end of year exams to be allowed to continue to the final. The class of degree that was awarded depended on the open note final exam and the repor…

Not too dissimilar for me at Birmingham, we had tutorials ~weekly. There were weekly problem sheets that counted for 10% of the grade though.

Similar re: exams, they were available but sticking rigidly to them didn’t help much.

Re: Do AI detectors work? Students face false cheating accusations

#825

Post by hydrolox »

youoy wrote: post #474
mikeyinternews wrote: post #458

One of my kid's teachers sent out a warning to students that all essays would be checked with AI detection software and the repercussions one would face if caught. A classmate did an AI check on the teacher's warning and it came back positive for having been AI-generated.

While I understand the spirit of your message, you should not care about that. "One of my kid's teachers set out a warning to students that all essays would be checked against the other students' essays to see if they are the same and the repercussions one would face if caught. A classmate did a Google search and found the questions of the essay as examples on a book." One thing is perfectly valid, the other one is n…

I understand your point, but I would say that it is not particularly appropriate for a teacher to use AI (or plagiarize, to an extent) in this context. Taking questions from an existing bank, in my opinion, is different to AI generating your prompt/email/etc. What I mean is, students will NOT listen even more so if they find out how blatantly hypocritical a teacher is being (in the hypothetical situation that the teacher really did use AI)

This isn't a made up situation. Teachers at my school have used AI for essay prompts, test questions, etc and it spreads around and generally leads to the sentiment that "if the teacher is doing it, they can't in good faith tell me to not". Imagine if in math class the teacher , after just telling the students they can't use a calculator, types in a simple arithmetic expression into their calculator.

Re: Do AI detectors work? Students face false cheating accusations

#826

Post by methodical »

sa46 wrote: post #805
methodical wrote: post #756

Earlier quoted context omitted.

I think the best test for a Junior is to ask them to submit some of their OSS or personal fun projects they've worked on. From my perspective, especially with Juniors who aren't expected to be extremely knowledgeable, displaying a sense of curiosity and a willingness to learn is much more important. If, hypothetically, there's two candidates, one who is more knowledgeable but has no personal projects versus someone w…

Most engineers, including good ones, that I've interviewed have no interesting GitHub contributions. GitHub is also game-able. Bootcamps, in particular, push their graduates to build an interesting GitHub portfolio. I've found that talking through projects is a weak indicator of competence. It's much easier to memorize talking points than to produce working code.

It may be a result of personal preference, but I struggle to see how talking through challenges encountered with a personal project are a poor indicator of competence. If you ask some boilerplate list of questions, sure, but few if any candidates could memorize all of the random in-the-weeds architecture questions one could ask while talking through someone's project. For a junior specifically, even a non-answer to these questions provides valuable insight into their humility and self-awareness. I also think that it'd be pretty easy to visually weed out personal projects created for the sake of saying one has personal projects, like a bootcamp may push to create, versus an actual passion project, and even easier to weed out during any actual discussion. I suppose YMMV, but in my experience, the body language and flow of discussion are vastly different when someone is passionate about a subject versus not.

Re: Do AI detectors work? Students face false cheating accusations

#827

Post by MerManMaid »

aniviacat wrote: post #613
2OEH8eoCRo0 wrote: post #544

Earlier quoted context omitted.

> The main courses are mandatory in order to obtain the degree. You can't "not sign up" for linear algebra if it's in your curriculum. The course might be mandatory but which professor you choose isn't. What if multiple professors teach it? Word gets around and everyone chooses the easy profs.

I studied for a popular degree at one of the largest universities in Germany. I never had a course be taught by multiple professors. If a course had many attendants, the room just got bigger. But that's just my personal experience. I don't know if it's different at other large universities.

In smaller countries like Germany increasing the class size makes sense but countries like the US, it just doesn't scale. Just to give a better sense, my quick google-fu (so take it with a grain of salt) shows Germany having 2.8M people actively enrolled in college vs the US with 18.1M.

So roughly 6x the amount of students.

Re: Do AI detectors work? Students face false cheating accusations

#828

Post by com2kid »

skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

> The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. But any teacher doing this will end up with no students signing up for their class. The only solution I see is the Higher Learning Commission mandating this for all classes.

Just one generation ago this was the norm. The only differences between how exams were given in my math classes were what size of note paper was allowed.

In general students hated the few classes that tried to use online platforms for grading, the sites sucked so much that students preferred pen and paper.

Also, it is a math class! The only thing that is needed is arguably a calculator, a pencil, and some paper. What the hell kind of technology are students using in class?

> The only solution I see is the Higher Learning Commission mandating this for all classes.

Colleges used to all have tech requirements, the big debate was to allow calculators with CAS or not.

> If we did truly devise a system that prevents cheating we’ll see that a very high percentage of current college students are not ready to be truly college educated.

What the heck are students doing in college then? I was paying good $$$ to go to college, I was there because I wanted to learn. Why the hell would I pay thousands of dollars to go to class and then not learn anything in the class, that would be a huge waste of my time!

Re: Do AI detectors work? Students face false cheating accusations

#829

Post by borski »

crummy wrote: post #103
borski wrote: post #25

Earlier quoted context omitted.

I don’t know how to break this to you, but cheating was always free, simple, and ubiquitous. Sure, ChatGPT wouldn’t write your paper; but your buddy who needed his math problem solved would. Or find a paper on countless sites on the Internet.

That wasn't free; people would charge money to write essays, and essays found online would be detected as such.

> essays found online would be detected as such

Uh huh. Except for the very very very high frequency with which they weren't.

Also, people traded favors. I'll grant you that's not technically "free," but it may as well have been.

Re: Do AI detectors work? Students face false cheating accusations

#830

Post by itronitron »

account42 wrote: post #603
Jcampuzano2 wrote: post #574

Earlier quoted context omitted.

It seems more strange in my opinion that you'd never have a course thats popular enough that more than one teacher holds sessions for it. You don't have the choice to not take the class, you just have choice with which professor you would like to take it with. And often you would have to get lucky anyway, since that session may be filled so you'd have to take it with the "harder" teacher anyway. For example with the…

Why would a university need multiple professorts seaching the same subject at the same time? A professor isn't a school teacher that needs to look after each student individually. And even for questions andexcercises those are often already handled by teaching assistants of which there can be many as needed. Having the choice between different professors with supposedly different difficulties for what is supposed to…

As absurd as having them teach the same course year after year? Why not then just record their lectures and place them online.
Post reply on HN