Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

Post reply on HN
Print view View original

881–890 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#881

Post by odo1242 »

mitthrowaway2 wrote: post #649
CBLT wrote: post #639

Earlier quoted context omitted.

Getting rid of grading sounds crazy, but it's actually happening. Los Angeles Unified, the second largest school district in America, is moving to "equitable grading", which amounts (imo) to pass/fail with extra pageantry. Teachers are being retrained _right now_ to equitable grading. I know an equitable grading champion at an LAUSD school, I'll see if I can get material to share. EDIT: I just received [0][1][2][3].…

In a pass/fail system, what does a student need to do for a teacher to be willing to fail them? What is the minimum bar to pass?

One of my teachers implemented a system like this. What they ended up doing was making it so that you had to score a (effectively) 9/10 on major assignments to pass the class (minor assignments were graded on completion), but had an infinite number of revisions with which to get this grade with feedback being provided each time you tried. Pretty much everyone passed, with more work required from some than from others. The only issue it ran into was with the final paper, where you (realistically) only had time to receive and make one to two revisions before the end of the semester and the deadline to submit grades.

Re: Do AI detectors work? Students face false cheating accusations

#882

Post by vineyardlabs »

alasdair_ wrote: post #790
stanford_labrat wrote: post #663

Earlier quoted context omitted.

Ironically enough, our lecture halls were simply not big enough. The space capped out at around 300-600 people and for popular topics such as programming 101 every semester would easily have 1500+ enrolled.

The difference is that in Europe, you apply to take a specific subject at university, like Computer Science, and there are only so many spots available so that effectively caps the class sizes. You don't have a bunch of other people taking the class that are not working on that specific degree.

This is also the case in the US. The majority of college courses are limited to people within a given major and can't be taken by outside majors with limited exceptions.

Re: Do AI detectors work? Students face false cheating accusations

#883

Post by dot5xdev »

jillesvangurp wrote: post #155

I'd expect smart people to be able to use tools to make their work easier. Including AI. The bigger picture here is that the current generation of students are going to be using and relying on AI the rest of their careers anyway. Making them do things the old fashioned way is not a productive way to educate them. The availability of these tools is actually an opportunity to raise the ambition level quite a bit. Unive…

> I would encourage students to use all the tools available to them. Especially AI. I'd set the bar pretty high.

How would you set the bar pretty high? How would you avoid just evaluating ChatGPT, instead of the actual student?

Re: Do AI detectors work? Students face false cheating accusations

#884

Post by aftbit »

greenavocado wrote: post #607
aftbit wrote: post #599

I haven't seen this discussed as much as I expected - is this even possible? Can a tool be built to - in general - determine if an LLM was used to generate text? Can a human even do it in every case? _Maybe_ you can detect default ChatGPT-3.5 responses. But if a student does a bit of mucking around with fine-tunes on local llama or uses a less-common public model, can you still tell? I have a similar question for AI…

The only way to be sure a student isn't cheating is to search them before they enter a secure room with nothing in it besides the student, the proctor, some paper, maybe some furniture, and proctor-provided pens or pencils to take an oral or written exam. In this age you can only truly judge a student's mind by observing their synthesis skills in-person.

I agree, but I'll argue that this is not responsive to my question, nor a reasonable goal in general. You cannot be _sure_ that a student isn't cheating without taking draconian measures, but you can likely catch a lot of lazy cheaters by applying imperfect methods. The problem comes when the methods are treated as infallible and there is no appeal process.

Re: Do AI detectors work? Students face false cheating accusations

#885

Post by dpkirchner »

dot5xdev wrote: post #83
gradus_ad wrote: post #12

Seems like the easy fix here is move all evaluation in-class. Are schools really that reliant on internet/computer based assignments? Actually, this could be a great opportunity to dial back unnecessary and wasteful edu-tech creep.

Moving everything in class seems like a good idea in theory. But in practice, kids need more time than 50 minutes of class time (assuming no lecture) to work on problems. Sometimes you will get stuck on 1 homework question for hours. If a student is actively working on something, yanking them away from their curiosity seems like the wrong thing to do. On the other hand, kids do blindly use the hell out of ChatGPT. It…

Minor quibble here: If a student gets stuck on a single homework problem for hours, they're probably hopelessly lost and would benefit from being interrupted. That or the problem is way too broad to be mere homework.

Re: Do AI detectors work? Students face false cheating accusations

#886

Post by nitwit005 »

pj_mukh wrote: post #669
lumost wrote: post #611

Earlier quoted context omitted.

My personal take, we’ve made the cost of failure to high and cheating too easy. As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating. As someone who studied physics and came out with a 2.7 GPA due to studying what I wanted…

Serious question from someone who is regularly tasked with hiring Juniors. What IS a good assessment for entry-level/right out of college positions? -> GPA can be gamed, as laid out. -> Take Home assessments can mostly be gamed, I want to assess how you think, now which tools you use. -> Personality tests favor the outgoing/extroverts -> On-location tests/leet code are a crapshoot. What should be best practice here?…

You listed out what you think the options are. You have to pick one, so pick the least bad.

Realize there are practical limits to knowledge here. In the case of a new graduate, they are likely to have little or no job experience, so no one actually knows how they function in a workplace. Even if they were a personal family friend who you knew quite well, there would be considerable uncertainty.

Re: Do AI detectors work? Students face false cheating accusations

#887

Post by recursive »

A4ET8a8uTh0 wrote: post #456
lupire wrote: post #439

Earlier quoted context omitted.

He's paying for the degree and the professional network. Studying would be a waste of time.

I hope it will not sound too preachy. You are right in a sense that it is what he thinks he is paying for, but is actually missing out on untapped value. He will not be able to discuss death as a concept throughout the lens of various authors. He will not wrestle with questions of cognition and its human limitations ( which amusingly is a relevant subject these days ). He will not learn anything. He is and will remai…

It's not wasteful if: the student "values" the credential but not the learning. Some people have no appetite for philosophy or the perspective of notable authors on death. He will probably learn something though.

Your characterization of an adult child does not seem fair. What makes someone an adult? If it's academic discourse, then why is it valuable?

I mean, if you're into it, more power to you. Go nuts with finally figuring out what makes a human. Just don't claim it's more virtuous than anyone else's hobby, unless you can find a reason.

To call it "wasteful" says that something of "value" is being squandered, but the value is perceived by each of us differently.

Re: Do AI detectors work? Students face false cheating accusations

#888

Post by recursive »

dudu24 wrote: post #846
jerf wrote: post #461

Earlier quoted context omitted.

The default tone of ChatGPT and the default tone of school or academic writing (at all levels) are not exactly the same, but in the grand vector space of such things, they are awfully close to each other. And all the LLMs have presumably already been fed with an awful lot of this sort writing, too. It's not a surprise that a by-the-numbers report, either in high school or college, of the sort that generally ought to…

> grand vector space what.

In the language of "embeddings" of machine learning.

Re: Do AI detectors work? Students face false cheating accusations

#889

Post by strken »

pj_mukh wrote: post #669
lumost wrote: post #611

Earlier quoted context omitted.

My personal take, we’ve made the cost of failure to high and cheating too easy. As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating. As someone who studied physics and came out with a 2.7 GPA due to studying what I wanted…

Serious question from someone who is regularly tasked with hiring Juniors. What IS a good assessment for entry-level/right out of college positions? -> GPA can be gamed, as laid out. -> Take Home assessments can mostly be gamed, I want to assess how you think, now which tools you use. -> Personality tests favor the outgoing/extroverts -> On-location tests/leet code are a crapshoot. What should be best practice here?…

What do you want out of your junior engineers? What is the actual skill, talent, or trait?

I don't think GPA, take-home assignments plus an interview about them, personality tests, or on-location tests like leetcode or architecture interviews are measuring the same thing. Are you just looking for any means to winnow down the pool of applicants, or is there an underlying ability you're searching for?

Re: Do AI detectors work? Students face false cheating accusations

#890

Post by FigurativeVoid »

adamc wrote: post #795
FigurativeVoid wrote: post #766

Earlier quoted context omitted.

> My personal take, we’ve made the cost of failure to high and cheating too easy. I agree with the first part, but I think the second follows from it. Take a class like organic chemistry. When I was in school, the grade was based on 5 exams, each worth 20% of your grade. Worse still, anything less than an A was seen as a failure for most students dreaming of medical/vet school. Of course you are going to have people…

The consequences of cheating could be made much more severe. I am troubled by this argument because it suggests people have no ethical core. If that is true then we are going to have problems with them regardless.

I think if you asked people who cheat if it was ethically wrong, they would admit it cheating is indeed unethical.

But we are really great about rationalizing away ethical issues. I suspect is a good grade is worth more than a personal sense of ethics.

As much as a med school wants ethical students, they want students with 4.0s more.

Post reply on HN