Do AI detectors work? Students face false cheating accusations • Hacker News Board

Re: Do AI detectors work? Students face false cheating accusations

#661

Post by injidup » Mon, Oct 21, 2024, 4:15 PM UTC

strogonoff wrote: post #269

injidup wrote: post #239

Earlier quoted context omitted.

Little Johnny who tried really hard but still can barely write a for loop doesn't deserve a place in a comp sci course ahead of little Timmy who for some reason thinks in computer code. Timmy might be a lazy arse but he's good at what he does and for minimal effort the outcomes are amazing. Johnny unfortunately just doesn't get it. He's wanted to be a programmer ever since he saw the movie Hackers but his brain just…

My evaluation: 1. Whoever determined that he does not “deserve” this is wrong. There may be other constraints, but no one gets to frame it as “deserves” when a child wants to learn something. 2. If a teacher is unable to teach Johnny to write a for loop, despite Johnny’s genuine utmost motivation , I would question teacher’s competence or at least fit. 3. Like any mentor, a professor in higher ed may want to choose w…

It's funny. You have created yourself a paradox. Replace comp sci with being a teacher. You have made the claim now that teachers can be incompetent but Johnny cannot be. Let's say Johnny wants to become a teacher and puts in lots of effort but just cannot teach. Now he is an incompetent teacher but at what point did he go from being judged on effort to being judged on ability? When he wanted to be a teacher and got a free pass for being a bad teacher? When he went for his first job and got a free pass for failing his exams? When his entire class learned nothing because he was unable to teach even though he put in lots of effort?

Where is the transition? At some point ability is more important than effort.

Re: Do AI detectors work? Students face false cheating accusations

#662

Post by hbn » Mon, Oct 21, 2024, 4:16 PM UTC

zahlman wrote: post #601

skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

Absolutely true, and not limited to the USA either. In university I can recall a computer graphics course where literally everyone got 100+% on problem sets (there were bonus questions of course) and the median score on the midterm was below 50%. Leading up to the exam I remember the prof leading an exam prep session, opening the floor to questions, and getting a sincere request from one of the students to please go…

In 2018 I did a 400-level CS class that was an introduction to computer audio. One of the assignments was to implement a fast fourier transform. After class I went to the cafeteria and hacked one out in like an hour or 2. A week or so later as the assignment was nearing due, apparently many, if not most of the students complained the assignment was too hard because... they seemingly just didn't know how to write code?

They ended up changing the assignment to where you could just find an implementation of a FFT online and write about it or something.

That's not even getting into the students who copy-pasted Wikipedia straight into their papers in that same class.

Re: Do AI detectors work? Students face false cheating accusations

#663

Post by stanford_labrat » Mon, Oct 21, 2024, 4:17 PM UTC

aniviacat wrote: post #613

2OEH8eoCRo0 wrote: post #544

Earlier quoted context omitted.

> The main courses are mandatory in order to obtain the degree. You can't "not sign up" for linear algebra if it's in your curriculum. The course might be mandatory but which professor you choose isn't. What if multiple professors teach it? Word gets around and everyone chooses the easy profs.

I studied for a popular degree at one of the largest universities in Germany. I never had a course be taught by multiple professors. If a course had many attendants, the room just got bigger. But that's just my personal experience. I don't know if it's different at other large universities.

Ironically enough, our lecture halls were simply not big enough. The space capped out at around 300-600 people and for popular topics such as programming 101 every semester would easily have 1500+ enrolled.

Re: Do AI detectors work? Students face false cheating accusations

#664

Post by skhunted » Mon, Oct 21, 2024, 4:17 PM UTC

dragonwriter wrote: post #638

skhunted wrote: post #587

Earlier quoted context omitted.

Around 50% of higher education in the United States is done at community colleges. Tuition accounts for 2/3 of our budget. State subsidy for 1/3. In the past the numbers were reversed. Enrollment in higher education went through a decade long decline. It is now the case that colleges are chasing tuition dollars. Students are the client.

> Around 50% of higher education in the United States is done at community colleges. Sure, but they don't set the rules; sure, they do much of the education, but much of the demand for them comess from bachelor’s-degree bound students, so the course selection is set by what bachelor’s degree granting institutions accept.

Matriculation agreements are based on content covered not on whether or not the students learned the material. But since community colleges have less grade inflation than other institutions passing our classes means more.

The Higher Learning Commision is a farce. It’s purpose is for sinecures for its employees.

Re: Do AI detectors work? Students face false cheating accusations

#665

Post by [deleted] » Mon, Oct 21, 2024, 4:18 PM UTC

chatmasta wrote: post #608

zahlman wrote: post #601

Earlier quoted context omitted.

Absolutely true, and not limited to the USA either. In university I can recall a computer graphics course where literally everyone got 100+% on problem sets (there were bonus questions of course) and the median score on the midterm was below 50%. Leading up to the exam I remember the prof leading an exam prep session, opening the floor to questions, and getting a sincere request from one of the students to please go…

In my algorithms class (and some others), our professor openly approved of collaboration on problem sets. He knew that students were going to collaborate anyway, so it may as well be encouraged and used as a pedagogical tool. The problem sets were more difficult because of this, but nobody was afraid to talk about them and help each other work through the proofs. The midterm and final exam were in-person in bluebooks…

[deleted]

Re: Do AI detectors work? Students face false cheating accusations

#666

Post by _heimdall » Mon, Oct 21, 2024, 4:18 PM UTC

visarga wrote: post #648

SketchySeaBeast wrote: post #493

Earlier quoted context omitted.

And we could get an AI to review the recording!

It's what OpenAI does. They have a small safety model checking on the big model.

That's OpenAI's current answer to safety. Its far too early to say whether they is actually a good approach to LLM safety.

Re: Do AI detectors work? Students face false cheating accusations

#667

Post by Spivak » Mon, Oct 21, 2024, 4:18 PM UTC

mitthrowaway2 wrote: post #649

CBLT wrote: post #639

Earlier quoted context omitted.

Getting rid of grading sounds crazy, but it's actually happening. Los Angeles Unified, the second largest school district in America, is moving to "equitable grading", which amounts (imo) to pass/fail with extra pageantry. Teachers are being retrained _right now_ to equitable grading. I know an equitable grading champion at an LAUSD school, I'll see if I can get material to share. EDIT: I just received [0][1][2][3].…

In a pass/fail system, what does a student need to do for a teacher to be willing to fail them? What is the minimum bar to pass?

At most universities you can talk most classes pass/fail by choice which means A-D is pass and F is fail.

The nice thing about an all pass/fail system is you can formalize the 'new' way grades are actually done in which A means meets expectations and anything less means did not. Making pass mean A/B takes a lot stress off students and C/D is already failing for practical purposes as often you can't continue with less than a B.

Re: Do AI detectors work? Students face false cheating accusations

#668

Post by lacker » Mon, Oct 21, 2024, 4:21 PM UTC

lumost wrote: post #611

skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

My personal take, we’ve made the cost of failure to high and cheating too easy. As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating. As someone who studied physics and came out with a 2.7 GPA due to studying what I wanted…

Employers need to wake up to this in hiring, too. You can get a 4.0 with a degree in computer science from a top school, and still not be able to program at all.

Some organizations still hire software engineers just based on resume and a nontechnical interview. This can easily be a disaster! You need to do a real assessment during the interview of how well software engineers can code.

Re: Do AI detectors work? Students face false cheating accusations

#669

Post by pj_mukh » Mon, Oct 21, 2024, 4:21 PM UTC

lumost wrote: post #611

skhunted wrote: post #491

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

My personal take, we’ve made the cost of failure to high and cheating too easy. As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating. As someone who studied physics and came out with a 2.7 GPA due to studying what I wanted…

Serious question from someone who is regularly tasked with hiring Juniors. What IS a good assessment for entry-level/right out of college positions?

-> GPA can be gamed, as laid out.

-> Take Home assessments can mostly be gamed, I want to assess how you think, now which tools you use.

-> Personality tests favor the outgoing/extroverts

-> On-location tests/leet code are a crapshoot.

What should be best practice here? Ideally something that controls for first-time interviewer jitters.

Re: Do AI detectors work? Students face false cheating accusations

#670

Post by Ukv » Mon, Oct 21, 2024, 4:21 PM UTC

blibble wrote: post #641

Ukv wrote: post #347

Earlier quoted context omitted.

> [...] but a viable model where students and laypeople have access to it? To be seen. You can run GPT-4-equivalent models locally. Even if all software and hardware advancements immediately halt, models at the current level will remain available.

how useful will a 2024 era model be in 2030? 2040? 2050?

A TI-84 Plus calculator (over 20 years old) is still useful today.

In isolation, I don't think a model necessarily becomes less useful over time. It'll still be as good at summarizing articles, translating text, correcting grammar, etc. for you as it is today.

If things do continue to advance and new models are released, which I think is likely, the old ones become less useful by comparison and in situations where there's competition. But then, through hardware/algorithmic improvements, better models also become feasible for universities/open-source groups/individuals - so you shouldn't be stuck with a 2024 era model.