Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

Post reply on HN
Print view View original

831–840 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#831

Post by com2kid »

skhunted wrote: post #596
_fat_santa wrote: post #576

Earlier quoted context omitted.

I remember taking a math class in college and the professor had a very unique way of dealing with cheating. He let us use our books, notes, and "any calculator capability" from our TI-84's. His rationale is that students will try to use these tricks anyways so just let them and then update the test to be "immune" from these advantages. Before every test he mentioned that we could use all those tools but always said "…

I’ve done this but then you end up with students who are not used to “thinking”. They do bad on the test. Now I’m known as a hard teacher. Now people avoid my classes. Administration hounds me for having s low passing rate. I need a job. I now give easy tests. The real issue as I see it is that no one wants to face the reality that far too many incapable, incurious people are going to college. So I pretend to give re…

When I was in college there were professors who were hard but fair, hard and not fair, and just easy.

Profs who were hard but fair never had a problem filling up their classrooms with students who self selected for wanting to learn.

The hard but not fair ones were just assholes IMHO.

The easy ones also had their classes filled up.

My community college had two history profs, one had all essay questions, one had multiple choice. The essay question prof was considered "hard", but so long as your essay justified your position and was well reasoned, you got full credit for the answer.

I hated the multiple choice prof. He gave the entire class his test bank every quarter and you just had to memory hundreds of questions and he'd pick 50 for the test. IMHO it took more time studying because I had to read the book and then memorize a bunch of pointless answers, vs reading the book and understanding what was going on, which I can typically do in the first pass.

Re: Do AI detectors work? Students face false cheating accusations

#832

Post by jasperry »

golergka wrote: post #720
skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

I never understood why americans do their exams with multi-option tests. Even if you don't cheat, these tests don't actually test knowledge, just memoization. For me a proper exam is when you get a topic, spend 30 minutes in a classroom preparing, and then sit down with an examiner to tell him about this topic and answer all the follow-up questions. We don't do multi-option tests at software interviews, and for a goo…

A big reason is that it's quicker and more objective to grade, making the heavy workload of teachers a little easier to shoulder.

I don't completely agree that multiple-choice questions can't test real knowledge. It is possible to write multiple-choice questions that require deep thinking and problem solving to select the correct answer (modulo a 25% chance of getting it right with a guess.)

It's true that MC questions can't evaluate the problem-solving process. You can't see how the student thought or worked through the problem unless you have them write things out. But again, that's a tradeoff with the time it takes to evaluate the students' responses.

Re: Do AI detectors work? Students face false cheating accusations

#834

Post by skhunted »

rincebrain wrote: post #802
skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

Honestly, the problem is not the cheating, per se. The problem is the lack of learning the material. You don't, IMO, directly care how they produced the answer, you care about it only as a proxy for them learning the material well enough to solve the problem. And making people do them in person with no technology is unrealistic - not because it can't be done, but because at that point, it's not a reflection of how yo…

You don't, IMO, directly care how they produced the answer, you care about it only as a proxy for them learning the material well enough to solve the problem.

I don’t care about the answer. I care about the thought process that went into finding the answer. The answer is irrelevant.

And making people do them in person with no technology is unrealistic - not because it can't be done, but because at that point, it's not a reflection of how you'd use it outside of that classroom, and people are going to call it pointless, and IMO they'd be right.

There’s body of knowledge a person trained in a given area ought to know without use of computers or notes. There are things a person who calls themself “an engineer” or “a physicist” ought to know off the top of their head. A person going into mechanical engineering ought to have some familiarity with how to integrate without using a computer. Such is my belief.

Re: Do AI detectors work? Students face false cheating accusations

#835

Post by com2kid »

itchyouch wrote: post #653
skhunted wrote: post #596

Earlier quoted context omitted.

I’ve done this but then you end up with students who are not used to “thinking”. They do bad on the test. Now I’m known as a hard teacher. Now people avoid my classes. Administration hounds me for having s low passing rate. I need a job. I now give easy tests. The real issue as I see it is that no one wants to face the reality that far too many incapable, incurious people are going to college. So I pretend to give re…

People want to be engaged with the work they believe in. Students or adults. Fundamentally, kids that are just trying to pass a class don't see the value in learning and it seems that the contributions towards the "pointless" school work are parts teacher attitudes, parts curriculum design, parts real-life applicability to the student's interests, parts framing. We've been using tests and such for far too long as a p…

> Fundamentally, kids that are just trying to pass a class don't see the value in learning and it seems that the contributions towards the "pointless" school work are parts teacher attitudes, parts curriculum design, parts real-life applicability to the student's interests, parts framing.

It is 100% societal. It is because society is focused on "get degree, get job, get money". It is because Western societies have gotten so damn competitive that if you don't succeed at any of the above, there is a non-trivial chance you won't even be able to afford a house to live in.

In America, I'll admit that No child left behind made it a lot worse, with tests left and right, which gives students the wrong impression of what learning is about.

Every class should be about critical thinking. Every single class. Multiple choice tests are a societal cancer and should be limited to a tiny fraction of tests given.

The point of school is to learn how to learn. That is it. What facts are taught are almost irrelevant. The point is to learn HOW to learn. Be that researching the history of fabric dyes in Ancient Egypt or making a scale drawing of one's house.

The "WHAT" IS NOT IMPORTANT.

The HOW is important.

How to write an essay, the topic doesn't matter.

How to learn about the culture of a country.

How to learn a new field of mathematics.

How to learn a new type of art.

How to give a presentation.

How to learn a hard science.

Yes the basics of physics and chemistry and such need to be taught. But the things that are learned should be inline with teaching the all important skill of how to learn.

Re: Do AI detectors work? Students face false cheating accusations

#836

Post by dragonwriter »

dragonwriter wrote: post #734
sqeaky wrote: post #234

Earlier quoted context omitted.

I can't think of a single time that we've ever willingly put down a technology that a single person could deploy and appear to be highly productive. You may as well try to ban fire. Looking at some of the most successful historical pushbacks against technology, taxes and compensation for displaced workers is about as much as we can expect. Even trying to put restrictions on AI is going to be very practically challeng…

> But I think the most basic of restrictions like mandating watermarks or tracing material of some kind in it might be possible and really that might do a lot to mitigate the worst problems. Watermarking output (anything that is detectable that is part of the structure of the text, visual--if even imperceptible--image, or otherwise integrated into whatever the primary output is) will make it take a bit more effort to…

And also, mandates for either are mainly going to effect use of public, hosted services; but the proliferation of increasingly-capable open models where fine-tuning and inference can be done locally on consumer hardware will continue and be an additional problem for anything that relies on such a mandate.

Re: Do AI detectors work? Students face false cheating accusations

#837

Post by light_hue_1 »

michaelt wrote: post #301
light_hue_1 wrote: post #277

Earlier quoted context omitted.

Oh yes. When I'm teaching a class of 200 students it's totally plausible that we're going to do 10 15 minute one on one conversations with every student. Because that's only 20 days non stop with no sleep. We would need to increase the amount of teaching staff by well over 10x to do this. The costs would be astronomical.

I said one conversation per student per class, and ten classes per year. Not 10 conversations per class per student. > The costs would be astronomical. Those 200 students have paid the college $549,600 for your class. The costs are already astronomical. Is it so unreasonable for some of that money to be spent on providing education?

I can't express how out of touch with reality this reply is.

The students paid me nothing. The university provides some TAs, that's it. But even if they gave me all of that money in cash to spend, this would be totally impossible.

I'm supposed to grade a student based on 1 conversation? Do you know how grading and teaching work? Can you imagine the complains that would come out of this process? How unfair it is to say that you have one 15 minute shot at a grade?

But fine, even if we say that I can grade someone based on 1 conversation. What am I supposed to ask during this 15 minute conversation? Because if I ask the every student the same thing, they'll just share the questions and we're back to being useless.

So now I need to prep unique questions for 200 people? Reading their background materials, projects, test results, and then thinking of questions? I need to do that and review it all before every session.

Even with a team of TAs this would be impossible.

But even if I do all of this. I spend hours per student to figure out what they did and know. I ask unique questions for 15 minutes so that we can talk without information leakage mattering. You know what the outcome will be? Everyone will complain that my questions to them were harder than those that I asked others. And we'll be in office hours with 200 people for weeks on end sorting this out and dealing with all the paperwork for the complaints.

This is just the beginning of the disaster that this idea would be.

It's easy to sit in the peanut gallery and say "Oh, wow, why didn't my arm surgery take 10 minutes, they just screwed two bones together right?" until you actually need to do the thing and you notice that it's far more complex than you thought.

Re: Do AI detectors work? Students face false cheating accusations

#838

Post by itronitron »

itchyouch wrote: post #653
skhunted wrote: post #596

Earlier quoted context omitted.

I’ve done this but then you end up with students who are not used to “thinking”. They do bad on the test. Now I’m known as a hard teacher. Now people avoid my classes. Administration hounds me for having s low passing rate. I need a job. I now give easy tests. The real issue as I see it is that no one wants to face the reality that far too many incapable, incurious people are going to college. So I pretend to give re…

People want to be engaged with the work they believe in. Students or adults. Fundamentally, kids that are just trying to pass a class don't see the value in learning and it seems that the contributions towards the "pointless" school work are parts teacher attitudes, parts curriculum design, parts real-life applicability to the student's interests, parts framing. We've been using tests and such for far too long as a p…

Most first year classes for STEM courses at most universities are very large, highly impersonal, and from what I have seen, often taught by very poor communicators.

Students want to be engaged in their coursework, but the universities aren't there to encourage or support it.

Re: Do AI detectors work? Students face false cheating accusations

#839

Post by aaplok »

skhunted wrote: post #596
_fat_santa wrote: post #576

Earlier quoted context omitted.

I remember taking a math class in college and the professor had a very unique way of dealing with cheating. He let us use our books, notes, and "any calculator capability" from our TI-84's. His rationale is that students will try to use these tricks anyways so just let them and then update the test to be "immune" from these advantages. Before every test he mentioned that we could use all those tools but always said "…

I’ve done this but then you end up with students who are not used to “thinking”. They do bad on the test. Now I’m known as a hard teacher. Now people avoid my classes. Administration hounds me for having s low passing rate. I need a job. I now give easy tests. The real issue as I see it is that no one wants to face the reality that far too many incapable, incurious people are going to college. So I pretend to give re…

> I’ve done this but then you end up with students who are not used to “thinking”.

Then we need to teach them. You are doing the right thing for being a "hard" teacher, and it doesn't prevent you from also being known as a caring one.

From experience, acknowledging the students' difficulties with it and emphasising that it is because they were not taught how to think (as opposed to some innate inability to do maths) can go a long way.

Re: Do AI detectors work? Students face false cheating accusations

#840

Post by wnc3141 »

_ea1k wrote: post #624
lumost wrote: post #611

Earlier quoted context omitted.

My personal take, we’ve made the cost of failure to high and cheating too easy. As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating. As someone who studied physics and came out with a 2.7 GPA due to studying what I wanted…

This really can't be emphasized enough. Universities and the initial hiring process really optimize for a score and not for learning. Those could be, and sometimes are, correlated, but it isn't necessarily the case. Really focusing on stretching yourself necessarily means lower grades. Why is that penalized? TBH, in software engineering a lot of people with lower grades tutor the ones with 4.0 averages. The skillsets…

I wonder if we should take a look at how students, all paying tuition, have vastly unequal outcomes when it comes to job opportunities. Essentially there is a high scarcity of "good jobs" available to all but from the most selective universities.

Essentially when a scarcity increases, there will always be an imperfect heuristic of selection.

I guess this is more of a public policy area but it seems reasonable that anyone working full time should have access to economic security. Essentially cheating on university is the first symptom of lifetime of vastly unequal access to economic security.

Post reply on HN