Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

741–750 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#741

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

Well, during the end of the pandemic I had the misfortune of hear some engineers undergrads talking about on how would they supposed to pass classes now that they were going to be in person; apparently a lot of them were doing just "fine" on online classes and tests...

Re: Do AI detectors work? Students face false cheating accusations

#742
post #611

Earlier quoted context omitted.

My personal take, we’ve made the cost of failure to high and cheating too easy. As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating. As someone who studied physics and came out with a 2.7 GPA due to studying what I wanted…

Serious question from someone who is regularly tasked with hiring Juniors. What IS a good assessment for entry-level/right out of college positions? -> GPA can be gamed, as laid out. -> Take Home assessments can mostly be gamed, I want to assess how you think, now which tools you use. -> Personality tests favor the outgoing/extroverts -> On-location tests/leet code are a crapshoot. What should be best practice here?…

One classic approach is to over-hire and weed out. I find some form of this de-facto happens anyway, so managing more explicitly has some benefits.

Re: Do AI detectors work? Students face false cheating accusations

#743

It may be time to step back and ask "Why do students cheat?" I think the answer is "The Stakes" of getting a poor "grade" that follows you in and ranks you. Eliminate that, and tests become a valuable self assessment of where a student is. Teaches become partners in growth, not adversaries that can cause long term harm with a black mark.

Breath of sanity here. I wish I could upvote you twice.

Re: Do AI detectors work? Students face false cheating accusations

#744

Earlier quoted context omitted.

Most of the big professional sports already have this figured out. New college graduates have to compete for a spot at training camp. Hire them as temp contracts for two weeks to two months and let them play with the starting team.

Sounds roughly like an internship

And that is one of the best ways to hire new grads. Take the best of the crop of interns you've had.

Re: Do AI detectors work? Students face false cheating accusations

#745
post #611

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

My personal take, we’ve made the cost of failure to high and cheating too easy. As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating. As someone who studied physics and came out with a 2.7 GPA due to studying what I wanted…

The solution may also be not to make classes too hard. If, for example, your physics classes were of the same difficulty as the ones in my undergrad (easy to medium difficulty for the most part), then the 2.7 GPA is probably an accurate reflection of your abilities.

But if you went to a top university with brutal courses, and got a 2.7 GPA, then all I'm seeing is you're not elite material. The number otherwise does not help me one bit in evaluating you.

BTW, having spent a lot of time out of the US - it's still pretty laid back in the US. A person who is 2.7 GPA material in the US would simply not get admission in any decent university in some countries. And plenty of people in the US start all over at another institution and do well - something many countries don't allow (state funded, lack of resources, you have to move out of the way to let the younger batch in).[1]

[1] A good friend of mine totally flunked out of his university. He spent time off in the military. Then started all over at a new university. Got really high grades. Went to a top school for his PhD and is now a tenured faculty member.

Re: Do AI detectors work? Students face false cheating accusations

#746
post #668
post #611

Earlier quoted context omitted.

My personal take, we’ve made the cost of failure to high and cheating too easy. As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating. As someone who studied physics and came out with a 2.7 GPA due to studying what I wanted…

Employers need to wake up to this in hiring, too. You can get a 4.0 with a degree in computer science from a top school, and still not be able to program at all. Some organizations still hire software engineers just based on resume and a nontechnical interview. This can easily be a disaster! You need to do a real assessment during the interview of how well software engineers can code.

Also you can hire people with 20+ years of experience that also can't code (where people claim to be a software engineer). FizzBuzz was a real filter for a while. It has amazed me how some people where able to slide by in larger organizations for years and then switch (internally or to another company) when the competency mattered. You can make a whole career it!

Re: Do AI detectors work? Students face false cheating accusations

#747
post #709

Earlier quoted context omitted.

Serious question from someone who is regularly tasked with hiring Juniors. What IS a good assessment for entry-level/right out of college positions? -> GPA can be gamed, as laid out. -> Take Home assessments can mostly be gamed, I want to assess how you think, now which tools you use. -> Personality tests favor the outgoing/extroverts -> On-location tests/leet code are a crapshoot. What should be best practice here?…

It's hard, and interviewing is better suited to answering "nope, not you!" questions than "yes, you'll be a good fit." Onsite interviews with a range of approaches seem to be the best I've found over the years. As much as it pains me, things like fizzbuzz are still useful, because people still lie about their ability to program in languages. If you claim to know C very well and can't knock that out in 5 minutes, and…

> If you claim to know C very well and can't knock that out in 5 minutes, and it takes you 45 minutes of prompting, well, you don't know C usefully.

I recently had an interview and a "skill test" in C. It was proctored by the interviewer in-person. I had so many questions about the questions. It was like, what is the output of "some code" and while obvious, there were some questions where specific CPU architecture mattered:

    #include 

    int main() {
        unsigned int x = 0x01020304;
        unsigned char *c = (unsigned char*)&x;

        printf("First byte of x: 0x%02x\n", *c);
        return 0;
    }
I was like, what architecture are we running on here? So, I answered that "it depends" and explained how it would depend on the architecture. They came back and said that I didn't know C.

Sure, whatever. Probably dodged a bullet.

Re: Do AI detectors work? Students face false cheating accusations

#748

One of my kid's teachers sent out a warning to students that all essays would be checked with AI detection software and the repercussions one would face if caught. A classmate did an AI check on the teacher's warning and it came back positive for having been AI-generated.

You've omitted the most important part - what happened after? Had the reason prevailed? ;-)

I'm asking, because all this "AI" text-generation stuff isn't a technology problem. It's 101% a human problem.

Re: Do AI detectors work? Students face false cheating accusations

#749
post #612
post #351

Earlier quoted context omitted.

It even came with handwriting built in as authentication mechanism! AI detectors hate this secret! On a more serious note - US removed cursive from their curriculum almost two decades ago - something i cant wrap my head around as cursive is something the rest of the world(?) uses starting in middle school and onwards through the whole adult life.

Sorry but nobody uses "cursive" writing. People barely write once they leave education - they type. When they do write it's legible separate characters or it's unreadable scrawl.

My brother (mid-20s) still writes in cursive. He does so beautifully, completely legibly, and rapidly, at a faster rate than he, or the average person, can write non-cursive.

Re: Do AI detectors work? Students face false cheating accusations

#750
I think it's clear that AI detectors don't work. The only thing that may work is if a bunch of people use AI, their code will be identical or almost identical.

But overall as someone who teaches a computational class where students write code, I'm still at loss on how to deal with ChatGPT-cheating.

So far I'm using 1) referring to lecture notes when discussing possible methods to solve problems 2) converting problems to images to make it harder to copy-paste into chatgpt 3) Being stricter for code that is almost correct but does not run 4) Using hidden datasets, or input data-files that needs to be processed to figure the best solution

But it's all a bit futile I feel.

Post reply on HN