Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Training students to prove they're not robots is pushing them to use more AI

techdirt.com

Post reply on HN
Print view View original

121–130 of 183 posts

Re: Training students to prove they're not robots is pushing them to use more AI

#121

Post by ufo »

kayo_20211030 wrote: post #40

Perhaps we should not grade students on weekly, or other occasional, writing during the term or semester. How about going back to the old system where, apart from experimental lab work, nothing is graded until the end of the term? All weekly assignments should just be considered prep for one exam at the end of the term where the student has an opportunity to demonstrate mastery of the course's subject matter. They ca…

Students are very grade-motivated and unfortunately they rarely do the homework assignments if they are not worth points.

At-home coding projects, writing essays, etc also exercise different skiils than you can test for in a 2 hour written exam. It's unfortunate that due to rampant AI cheating, we can no longer reward the students who put in the work and develop these skills.

Re: Training students to prove they're not robots is pushing them to use more AI

#122

Post by thfuran »

kayo_20211030 wrote: post #81
thfuran wrote: post #72

Earlier quoted context omitted.

To what end? Not cheating on the weekly assignments is surely more beneficial to learning than cheating on them is, but I don’t see how removing the assignments altogether would help students learn.

If you nail the one exam, you get an A+. If you fail it, you get an F. In between, you get what your score says you get.

I understand the proposed grading system but not the reason for selecting that particular system.

Re: Training students to prove they're not robots is pushing them to use more AI

#123

Post by wongarsu »

lich_king wrote: post #83
Zigurd wrote: post #66

The core of the problem the article is about isn't AI or LLMs, it's about scam software that claims to catch cheating. It's crap for the same reasons that crime predictions software is crap. It's selling a panacea, and that kind of product inherently attracts scammers. If your school uses software to detect AI writing, that's a problem with the quality of your school. The people choosing that software are too stupid…

I'm always startled about how HN approaches these topics. When we have a press release from a university about how researchers can detect thoughts via fMRI, we have no issue with the claim. But if a vendor makes a pretty believable claim that there are repetitive statistical patterns in LLM output, it's all of sudden treated the same as palm reading. The problem isn't that AI detection doesn't work. State of the art…

You can detect if texts from a year ago used AI based on statistical patterns. Nobody is taking issue with that. But once you tell people "we will run these tests to detect if your future submissions are using AI" you create an adversarial environment and your statistical methods will continuously break. Not because statistics is broken, but because you are trying to hit a moving target that doesn't want to be hit.

That's not like detecting thoughts via fMRI, it's like detecting tomorrows malware with yesterday's malware signatures. Or like researchers making a vaccine against the common cold

And the obvious proposal to fix that has been made multiple times in this thread: don't make take-at-home tasks part of the grade. Instead of trying to punish what you can't reliably detect, take away the incentive to do it in the first place

Re: Training students to prove they're not robots is pushing them to use more AI

#124

Post by snyp »

varjag wrote: post #99
snyp wrote: post #84

Earlier quoted context omitted.

This approach does not really solve the core issue. In practice, students often do poorly when evaluation is concentrated in one end of term exam. It also pushes many students to cram at the end of the term instead of learning steadily. A better approach is to rethink what we assess and how we assess it. Research shows that the design of assessments plays an important role in academic integrity. Assignments that requ…

It's a noble motivation (and not even unrealistic in 2015 ) but what you'll get would still be generative output. If the only remedy is monitored end of term exams, so be it.

Err why can’t there be weekly monitored exams in class?

Re: Training students to prove they're not robots is pushing them to use more AI

#125

Post by cm11 »

ritlo wrote: post #56
Buttons840 wrote: post #22

Tangent: I've noticed I write a lot different because of combative online arguments. I have a problem. So much of my communication is directed to people who don't want to hear me or understand me. So I've become very punchy and repetitive, trying to hammer home ideas that people are either unable or unwilling to understand. I need to find ways to talk to people who want to hear and understand me. It's hard to find ot…

One of the worst habits distinctive to online discussion-board writing (especially the sorts of places with lots and lots of people and where it's fairly hard to get permanently kicked out—like here) is too much hedging and over-specifying to try to head off shitposting by bad or bad-faith readers. It's all over forum posts, and it's poor writing, but without moderation that slaps down responses based on plain mis-re…

I totally agree with this. I would add that it's well beyond the discussion boards. It's probably most clear there and it's well possible we learned it there and then took it into our social interactions everywhere, but the majority of my irl interactions—except with my closest friends—are sorta like this. Sometimes I think its ADHD, other times I think it could be any number of things, but I think to say anything that isn't dead simple (or in dead agreement with the other person), you need a few sentences. Often, you need to hear the third sentence before the first will make sense. But if you get distracted by the first one or can't suspend your disagreement enough to get to the third you will think the person is mistaken. You'll think that about both their first point and the larger one, which you didn't really hear or even get to but thought you did. So the speaker does the hedging each sentence in hopes of getting to the third (or whatever) sentence.

Re: Training students to prove they're not robots is pushing them to use more AI

#126

Post by ufo »

gmadsen wrote: post #114
snyp wrote: post #84

Earlier quoted context omitted.

This approach does not really solve the core issue. In practice, students often do poorly when evaluation is concentrated in one end of term exam. It also pushes many students to cram at the end of the term instead of learning steadily. A better approach is to rethink what we assess and how we assess it. Research shows that the design of assessments plays an important role in academic integrity. Assignments that requ…

then they fail. We are talking about adults.

Students are also people. If we're managing a software project, a single deadline at the end is sure to suffer from delays. It's better to split things into shorter deliverables with more frequent feedback.

Re: Training students to prove they're not robots is pushing them to use more AI

#127

Post by upboundspiral »

kayo_20211030 wrote: post #40

Perhaps we should not grade students on weekly, or other occasional, writing during the term or semester. How about going back to the old system where, apart from experimental lab work, nothing is graded until the end of the term? All weekly assignments should just be considered prep for one exam at the end of the term where the student has an opportunity to demonstrate mastery of the course's subject matter. They ca…

The purpose of grades is to punish students, something which they are keenly aware of. Remove grades from the equation and hold students back until they have mastered the material and they will cease cheating.

If someone knows 80% of the topics on an exam like the back of their hand and doesn't know the other 20% they shouldn't get a B, they should pass the subjects they know and be asked to retake and relearn the subjects they don't know.

When people know they can make mistakes and the result is not a perpetual black mark on their record (any grade not an A) but they are given the chance to improve and demonstrate this improvement then perhaps they might be more willing to admit and understand mistakes instead of cheating.

Re: Training students to prove they're not robots is pushing them to use more AI

#128

Post by kayo_20211030 »

abathur wrote: post #108
kayo_20211030 wrote: post #100

Earlier quoted context omitted.

That would be really poor course design :-)

There are many disciplines in which students work on effectively distinct projects. For example, the life-changingly-well-designed newswriting course I took in college assigned every single student a different story to spend several weeks reporting out so that we wouldn't all be out harassing the same poor people for interviews.

Genuinely interested. What was the final like? This seems more in the experimental science (ok, journalism) category. I may have to adjust my thinking to be more expansive and also include things like "vocational".

Re: Training students to prove they're not robots is pushing them to use more AI

#129

Post by wongarsu »

cm11 wrote: post #125
ritlo wrote: post #56

Earlier quoted context omitted.

One of the worst habits distinctive to online discussion-board writing (especially the sorts of places with lots and lots of people and where it's fairly hard to get permanently kicked out—like here) is too much hedging and over-specifying to try to head off shitposting by bad or bad-faith readers. It's all over forum posts, and it's poor writing, but without moderation that slaps down responses based on plain mis-re…

I totally agree with this. I would add that it's well beyond the discussion boards. It's probably most clear there and it's well possible we learned it there and then took it into our social interactions everywhere, but the majority of my irl interactions—except with my closest friends—are sorta like this. Sometimes I think its ADHD, other times I think it could be any number of things, but I think to say anything th…

To add to this: another sign of posting on online boards is starting your comments with "I agree" because otherwise the other person might default to assuming you are disagreeing (as is the norm for replies), leading to a comment chain of people violently agreeing with each other without realizing it

Re: Training students to prove they're not robots is pushing them to use more AI

#130

Post by Unearned5161 »

zahlman wrote: post #46
Unearned5161 wrote: post #38

Earlier quoted context omitted.

The point has always been the act of writing itself. What you write about is almost irrelevant; it’s that you spent the time writing, that you had ideas in your head, and that you squeezed them onto the page.

Sure. And my point is that the assignment is poorly conceived if an LLM's output can appear to "have ideas" that satisfy the prompt. Last I checked, they don't do a good job of modeling a specific, non-notable person within particular constraints, and then all the relevant life experiences of that person. An LLM essay should be human-detectable for the same reasons that one from an essay mill would be.

No matter how intricate and detailed an object is, it will appear similar to any other blurry mess if it's viewed through a shoddy lens.

I think your point stands for upper level work; however, at medium to lower levels, your counterfactual starts to weaken. The ideas have always been there, but it's the ability to express them--well enough to notice their presence--that is not.

Post reply on HN