Amazing article. Tech bros need to think less like machines.
Training students to prove they're not robots is pushing them to use more AI
141–150 of 183 posts
Re: Training students to prove they're not robots is pushing them to use more AI
#142nobody's asking who profits from false positives. these AI detection vendors have a direct financial incentive to flag aggressively. more flags = "more value" = more school contracts renewed. same playbook as selling antivirus to your grandma. sell fear, charge per seat, and make the false positive rate someone else's problem.
Do you have any evidence to back this up or is it speculative? My institution subscribes to TurnItIn's AI detector. The documentation is quite clear that the system is tuned in a manner that produces a significant number of false negatives and minimizes false positives. They also state that they don't report anything under "20% AI-generated" content. So the marketing I've seen is intended to reassure skittish adminis…
Fair point, the "tuned to flag aggressively" claim was speculative on my part. Turnitin's own documentation says they favor false negatives over false positives.
That said, their accuracy claims have been disputed before. Inside Higher Ed [1] reported that Turnitin's real-world false positive rate was higher than originally asserted, and the company declined to disclose the updated number. And, USD also noted that while Turnitin claimed Now, those are from 2023 and the product (and AI in general) has been updated drastically since. But the broader incentive problem holds even if the detector itself is conservatively tuned. The product is a black box. And the downstream cost of errors falls entirely on students, not on Turnitin's renewal rate. You don't need aggressive tuning for the incentive structure to be broken.
[1] https://www.insidehighered.com/news/quick-takes/2023/06/01/t...
[2] https://lawlibguides.sandiego.edu/c.php?g=1443311&p=10721367
Re: Training students to prove they're not robots is pushing them to use more AI
#143Earlier quoted context omitted.
then they fail. We are talking about adults.
Students are also people. If we're managing a software project, a single deadline at the end is sure to suffer from delays. It's better to split things into shorter deliverables with more frequent feedback.
You can have optional assignments and quizzes that serve that purpose.
If you take away the credit given for homework, you still can give feedback, while removing any incentive to cheat.
Re: Training students to prove they're not robots is pushing them to use more AI
#144Earlier quoted context omitted.
There are many disciplines in which students work on effectively distinct projects. For example, the life-changingly-well-designed newswriting course I took in college assigned every single student a different story to spend several weeks reporting out so that we wouldn't all be out harassing the same poor people for interviews.
Genuinely interested. What was the final like? This seems more in the experimental science (ok, journalism) category. I may have to adjust my thinking to be more expansive and also include things like "vocational".
Grammar and AP style rules, iirc. (I may not. It's been enough years now. I did try and fail to find the syllabus in my box of five-star notebooks. We mostly used reporters notebooks for this class, and I took it over the summer. The materials are probably in a plastic bag somewhere...)
Re: Training students to prove they're not robots is pushing them to use more AI
#145The core of the problem the article is about isn't AI or LLMs, it's about scam software that claims to catch cheating. It's crap for the same reasons that crime predictions software is crap. It's selling a panacea, and that kind of product inherently attracts scammers. If your school uses software to detect AI writing, that's a problem with the quality of your school. The people choosing that software are too stupid…
20 years ago (so very outdated), I TAed an introductory CS class for engineers who weren't going to be majoring in CS. We used MOSS [1]. Maybe it was the threshold we picked, but the results were pretty blatant. People renamed variables, renamed functions, changed comments, and the clever ones inlined or extracted a function. A lot forgot something, copying a bug or quirk from the original.
Re: Training students to prove they're not robots is pushing them to use more AI
#146Perhaps we should not grade students on weekly, or other occasional, writing during the term or semester. How about going back to the old system where, apart from experimental lab work, nothing is graded until the end of the term? All weekly assignments should just be considered prep for one exam at the end of the term where the student has an opportunity to demonstrate mastery of the course's subject matter. They ca…
Unfortunately this approach is known to be really bad for some students. Continuous assessment is bad for other students.
I was lucky in that my education last millennium was almost entirely make-or-break final exams, which suit me well. I’m bad at routinely completing little assignments, but shine under crazy pressure. Other students are the opposite.
We’ve tried both kinds over time and trend. Neither is perfect.
Re: Training students to prove they're not robots is pushing them to use more AI
#147Earlier quoted context omitted.
I don't disagree with you that a reasonable way to cope with the current problems is to ensure everything that "counts" is done in a controlled environment, but pedagogy and its goals are vast. There are things you learn from spending several days structuring a 20-page argument that you will not learn (and cannot assess) from oral examination or a 5-paragraph essay written in a blue book.
But there's no reason to expect that work to be graded. It should be a learning exercise which trains skills later tested under exam conditions.
Many students will simply not do these assignments. They should but they won’t. Continuous assessment partly solves this.
Re: Training students to prove they're not robots is pushing them to use more AI
#148Earlier quoted context omitted.
Students are also people. If we're managing a software project, a single deadline at the end is sure to suffer from delays. It's better to split things into shorter deliverables with more frequent feedback.
You can have optional assignments and quizzes that serve that purpose. If you take away the credit given for homework, you still can give feedback, while removing any incentive to cheat.
Few students do optional assignments unfortunately. Other tasks that are directly worth a gradetend to take priority (e.g. studying for another class that has an exam this week).
Re: Training students to prove they're not robots is pushing them to use more AI
#149Perhaps we should not grade students on weekly, or other occasional, writing during the term or semester. How about going back to the old system where, apart from experimental lab work, nothing is graded until the end of the term? All weekly assignments should just be considered prep for one exam at the end of the term where the student has an opportunity to demonstrate mastery of the course's subject matter. They ca…
This approach does not really solve the core issue. In practice, students often do poorly when evaluation is concentrated in one end of term exam. It also pushes many students to cram at the end of the term instead of learning steadily. A better approach is to rethink what we assess and how we assess it. Research shows that the design of assessments plays an important role in academic integrity. Assignments that requ…
> Assignments that require original thinking and regular engagement can reduce incentives to cheat and improve learning outcomes.
At some point in college when I was thinking about law school, I learned about the Socratic Method. It was weird because up to that point in college, I just pretty much flew under the radar and took exams. It was far different than high school, and I realized my high school did pretty much use the Socratic Method. It wasn't as intense as law school, but every class, maybe 4-5 people would we grilled by teachers. This was called "participation."
Shy? Anxiety? Yeah, that didn't matter. Your number would eventually be up a few times a month. You had to prepare and know the assignments, otherwise your grade would suffer and public humiliation was a real thing.
Re: Training students to prove they're not robots is pushing them to use more AI
#150Tangent: I've noticed I write a lot different because of combative online arguments. I have a problem. So much of my communication is directed to people who don't want to hear me or understand me. So I've become very punchy and repetitive, trying to hammer home ideas that people are either unable or unwilling to understand. I need to find ways to talk to people who want to hear and understand me. It's hard to find ot…
One of the worst habits distinctive to online discussion-board writing (especially the sorts of places with lots and lots of people and where it's fairly hard to get permanently kicked out—like here) is too much hedging and over-specifying to try to head off shitposting by bad or bad-faith readers. It's all over forum posts, and it's poor writing, but without moderation that slaps down responses based on plain mis-re…
>too much hedging and over-specifying to try to head off shitposting by bad or bad-faith readers.
yeah but if the OP doesn't do that and you confront their argument they can retreat into definitions and ambiguity without addressing your rebuttal. i think its good manners to be hyper-specific particularly on HN where there tend to be a lot of martian brained people who need it to engage with you. the fuzziness just won't do.