Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

791–800 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#791
post #58

That's kinda nuts how adult people learned to trust some random algorithms in a year or two. They don't know how it works, they cannot explain it, they don't care, it just works. It's magic. If it says you cheated, you cheated. You cannot do anything about it. I want to emphasize, that this isn't really about trusting magic, it's about people nonchalantly doing ridiculous stuff nowdays and that they aren't held accou…

> That's kinda nuts how adult people learned to trust some random algorithms in a year or two. They don't know how it works, they cannot explain it, they don't care, it just works. It's magic.

Well, you shouldn’t be so surprised. You just described 95%+ of the population’s approach to any form of technology. And there’s very rarely any discomfort with such ignorance, nor any desire to learn even the basics. It’s very hard to understand for me — some of us just have to know!

Re: Do AI detectors work? Students face false cheating accusations

#792

Earlier quoted context omitted.

There are institutions that still require a public defence for a PhD, not merely a viva. Oslo University for instance: https://www.uio.no/english/research/phd/

What PhD program doesn't require a public defense? I'm currently a PhD candidate, and our program includes separate written and oral qualifying exams during the first year or two, and a public defense of the dissertation at the end. I thought some minor variation of this was nearly universal. It's also my observation, by the way, that the public dissertation defense (and even the written dissertation itself) is less…

Imagine Alan Turing's defense being a summary of 3 papers. The actual issue is that advanced education is increasingly not about doing fundamental scholarship but a pipeline for (re)producing a clerisy-intellectual class. There are a lot of leftist academics who point out this sea change in academia over the last century, see for example Norm Finkelstein's remarks on this but there are others who talk about this.

Re: Do AI detectors work? Students face false cheating accusations

#793
post #438

Earlier quoted context omitted.

What about those of us who can explain our ideas and thinking clearly and in great detail in writing but would struggle to even prove we've heard of the topic orally?

I'm not going to sugar coat it and it may sound harsh, but I doubt this is ever truly an issue outside of the minute edge cases. Yes, there are people who have trouble with public speaking to a debilitating degree, but it would be excessively rare for someone to not at the very least in a one on one with their professor/teacher be able to be so badly affected as to not seem they've even heard of a topic or at least b…

This is the kind of opinion that should be common sense but is highly controversial in the modern educational climate, for whatever reason. Probably the whole, "You can't judge a fish by its ability to climb a tree" quote being misapplied constantly.

Re: Do AI detectors work? Students face false cheating accusations

#794
post #732

I’ve been teaching in higher education for 30 years and am soon retiring. I teach math. In every math course there is massive amounts of cheating on everything that is graded that is not proctored in a classroom setting. Locking down browsers and whatnot does not prevent cheating. The only solution is to require face-to-face proctored exams and not allow students to use technology of any kind while taking the test. B…

I did a "hard" degree and saw classmates who worked half as hard sail by me, because they cheated. Groups that share answer banks, in-class quizzes with answers shared (when they were not supposed to be), group projects that used last year's stuff. All of it, all the way through final exams, which people had answer keys to. I had a few classmates that were formally investigated for cheating by the university; their p…

You're supposed to work as hard as you can, then cheat for the grade.

Re: Do AI detectors work? Students face false cheating accusations

#795
post #611

Earlier quoted context omitted.

My personal take, we’ve made the cost of failure to high and cheating too easy. As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating. As someone who studied physics and came out with a 2.7 GPA due to studying what I wanted…

> My personal take, we’ve made the cost of failure to high and cheating too easy. I agree with the first part, but I think the second follows from it. Take a class like organic chemistry. When I was in school, the grade was based on 5 exams, each worth 20% of your grade. Worse still, anything less than an A was seen as a failure for most students dreaming of medical/vet school. Of course you are going to have people…

The consequences of cheating could be made much more severe.

I am troubled by this argument because it suggests people have no ethical core. If that is true then we are going to have problems with them regardless.

Re: Do AI detectors work? Students face false cheating accusations

#796
Essay writing is a waste of time. The biggest waste of time is the essays students turn out for college admissions. I mean really - think back to when you were 17-19 yrs old. What exactly was going through your mind - how you plan to change the world or where you want to hang out and what your next adventure is going to be?

For me it was the latter. Luckily, where I grew up in India the bogeyman was entrance exams. They are bad but in a way better than essays because you have a very clear expectation of success.

Either way, I hope GenAI finally makes essay writing obsolete so that we may move on to other better methods of assessing students. For those in flux as this situation rolls through - my sympathies. Educators have been lazy and you're paying the price.

Re: Do AI detectors work? Students face false cheating accusations

#797

Are any students coming up with a process to prove their innocents when they get falsely accused? If I was still in school I would write my docs in a Google Doc which provides the edit history. I could potentially also record video of me typing the entire document as well or screen recording my screen.

All of that still wouldn't prove that you didn't use any sorta LLM to get it done. The professor could just claim you used ChatGPT on your phone and typed the thing in, then changed it up a bit.

Guess you need to livestream it on twitch with multiple camera angles.

Re: Do AI detectors work? Students face false cheating accusations

#798
post #769

Earlier quoted context omitted.

Anything big endian. unsigned int x = 0x01020304; unsigned char *c = (unsigned char*)&x; Assume x is stored at 0x100. On a little endian architecture (x86, most modern ARM systems, etc), it will be stored in memory as [04][03][02][01], from bytes 0x100 to 0x103. If you assign char c to the address of x (0x100), it will read one byte, which is 0x4. However, on a big endian system, that same value would be stored in me…

Do you know if compilers are smart enough to return 04 even on big-endian architectures nowadays? For some reason I'm under the impression that (at least clang and gcc) are able to change this from "first byte in x" to "least significant byte in x" but don't actually know why I think that. Maybe embedded compilers typically don't?

To expand slightly on Syonyk said: the compiler cannot do it, because the object is stored between addresses c and c + sizeof(unsigned int). You can use this information to, for example, copy the object to another place with memcpy, and that of course wouldn't work if c wasn't pointing to the "leftmost" byte in the memory.

Unless, I suppose, sizeof was negative :).

Re: Do AI detectors work? Students face false cheating accusations

#799
post #347

Earlier quoted context omitted.

> [...] but a viable model where students and laypeople have access to it? To be seen. You can run GPT-4-equivalent models locally. Even if all software and hardware advancements immediately halt, models at the current level will remain available.

how useful will a 2024 era model be in 2030? 2040? 2050?

How useful is it to argue about what would happen in the extraordinarily unlikely eventuality that all LLM development will cease in 2024, wherein everyone with the proclivity to use LLMs will be stuck with exactly these same models for decades to come?

Re: Do AI detectors work? Students face false cheating accusations

#800
post #611

Earlier quoted context omitted.

My personal take, we’ve made the cost of failure to high and cheating too easy. As a student, the only thing the next institution will see is GPA, school, major. Roughly in that order. If the cost of not getting an A is exclusion from future opportunities- then students will reject exclusion by taking easier classes or cheating. As someone who studied physics and came out with a 2.7 GPA due to studying what I wanted…

Serious question from someone who is regularly tasked with hiring Juniors. What IS a good assessment for entry-level/right out of college positions? -> GPA can be gamed, as laid out. -> Take Home assessments can mostly be gamed, I want to assess how you think, now which tools you use. -> Personality tests favor the outgoing/extroverts -> On-location tests/leet code are a crapshoot. What should be best practice here?…

My process is as follows:

1. Live coding, in Zoom or in person. Don't play gotcha on the language choice (unless there's a massive gulf in skill transference, like a webdev interviewing for an embedded C position). Pretend the 13 languages on the candidate's resume don't exist. Tell them it can be any of these x languages, which are every language you the interviewer feel comfortable to write leetcode in.

2. Write some easy problem in that language. I always go with some inefficient layout for the input data, then ask for something that's only one or two for loops away from being a stupid simple brute force solution. Good hygienic layout of the input data would have made this a single hashtable lookup.

3. Run the 45 minute interview with a lot of patience and positive feedback. One of the best hires in our department had first-time interview nerves and couldn't do anything for the first 10 minutes. I just complimented their thinking-out-loud, laughed at their jokes, and kept them from overthinking it.

4. 80% of interviewees will fail to write a meaningful loop. For the other 20%, spend the rest of the time talking about possible tradeoffs, anecdotes they share about similar design decisions, etc. The candidate will think you're writing in your laptop their scoring criteria, but you already passed them and generated a pop-sci personality test result for them of questionable accuracy. You're fishing for specific things to support your assessment, like they're good at both making and reviewing snap decisions and in doing so successfully saved a good portion of interview time, which contributed to their success. If it uses a weasel word, it's worth writing down.

5. Spend an hour (yes, longer than the interview) (and yes, block this time off in your calender) writing your interview assessment. Start with a 90s-television-tier assessment. For example, the candidate is nimble, constantly creating compelling technical alternatives, but is not focused on one, and they often communicate in jargon. DO NOT WRITE THIS DOWN. This is the lesson you want the geriatric senior management to take away from reading your assessment. Compose relatively long (I do 4 paragraphs minimum) prose that describes a slightly less stereotyped version of the above with plenty of examples, which you spent most of the interview time specifically fishing for. If the narrative is contradicted by the evidence, it's okay to re-write the narrative so the evidence fits.

6. When you're done, skim the job description you're hiring for. If there's a mismatch between that and the narrative you wrote, change your decision to no hire and explain why.

Doing this has gotten me eye rolls from coworkers but compliments at director+ level. I have had the CTO quote me once in a meeting. Putting that in my performance review packet made the whole thing worth it.

Post reply on HN