Live data from Hacker News

Training students to prove they're not robots is pushing them to use more AI

techdirt.com

111–120 of 183 posts

Re: Training students to prove they're not robots is pushing them to use more AI

#111
post #16

This is what terrifies me about the public school system. A revolution has occurred, but it’s unevenly distributed. The schools simply don’t have the flexibility, agility, or frankly it seems motivation to adapt to what has already happened. The ship has sailed; essay writing is no longer a viable form of assessment. The idea to try to build a reliable AI detector is asinine, and fundamentally misunderstands how any…

Public Schools. I think terror there is built as a feature, not a bug. So be afraid.

But keep in mind, it may have always been this way. God bless those few cool teachers in each school who are aware of this and work to rescue a few who need it.

Love changes everything. Good teachers matter.

Re: Training students to prove they're not robots is pushing them to use more AI

#112

> The assignment had been to write an essay about Kurt Vonnegut’s Harrison Bergeron—a story about a dystopian society that enforces “equality” by handicapping anyone who excel Did not this self censorship process started decades ago? There are certain answers expected in academia, arguing for anything else would get you in troubles. Not using “devoid” seems pretty minor inconvenience. For me biggest wtf is why studen…

Avoid the theory-heavy disciplines. You won't be told what to think (as often) if you take History and Geography rather than Sociology and Gender Studies.

Re: Training students to prove they're not robots is pushing them to use more AI

#113

> The assignment had been to write an essay about Kurt Vonnegut’s Harrison Bergeron—a story about a dystopian society that enforces “equality” by handicapping anyone who excel Did not this self censorship process started decades ago? There are certain answers expected in academia, arguing for anything else would get you in troubles. Not using “devoid” seems pretty minor inconvenience. For me biggest wtf is why studen…

An essay is a good gauge of how one can organise their thoughts, argue a position, respond to a stimulus. In short it’s a good way measure thinking.

This -- and you might just learn how to conduct some research along the way.

Re: Training students to prove they're not robots is pushing them to use more AI

#114
post #84

Perhaps we should not grade students on weekly, or other occasional, writing during the term or semester. How about going back to the old system where, apart from experimental lab work, nothing is graded until the end of the term? All weekly assignments should just be considered prep for one exam at the end of the term where the student has an opportunity to demonstrate mastery of the course's subject matter. They ca…

This approach does not really solve the core issue. In practice, students often do poorly when evaluation is concentrated in one end of term exam. It also pushes many students to cram at the end of the term instead of learning steadily. A better approach is to rethink what we assess and how we assess it. Research shows that the design of assessments plays an important role in academic integrity. Assignments that requ…

then they fail. We are talking about adults.

Re: Training students to prove they're not robots is pushing them to use more AI

#115

Perhaps we should not grade students on weekly, or other occasional, writing during the term or semester. How about going back to the old system where, apart from experimental lab work, nothing is graded until the end of the term? All weekly assignments should just be considered prep for one exam at the end of the term where the student has an opportunity to demonstrate mastery of the course's subject matter. They ca…

Even before LLMs, there was a _lot_ of deception and cheating in university. I -- and I do not say this with pride -- used to write essays for my classmates for money. In my own defense, I needed the money. I also know that in addition to homework for money, many fraternities and sororities kept copies of prior exams and assignments, and getting access to these was one of the perks of membership. Knowing what kind of questions to expect (let alone the exact questions) can easily give someone a few extra IQ points for free.

Personally, I felt that the drive to automate the parts of the professors' workloads that mattered (i.e. teaching and grading and evaluation and research), only so that they can be given work that matters less the more they do it (i.e. publishing slightly different flavors of the same paper, to meet KPIs), was oddly perverse.

The multiple-choice test and the puzzle-solving test and really any standardized test can be exploited by any group that is sufficiently organized. This is also true in corporate interviewing where corporations think (or pretend) that they are interviewing an individual, whereas they are actually interviewing a _network_ of candidates who share details about the interviewers and the questions. I know people who got rejected in spite of getting all the interview questions correct (the theory is that nobody can do that well, so they must have had help from previously rejected/accepted candidates).

The word "trust" shares a root with the word "tree" and "truth" and "druid". Most exams and interviews are trying to speed-run trust-building (note that "verification" is from the latin word that means "true"). If trust and truth are analogous to "tree", then we are trying to speed-run the growth of a tree -- much like the orange tree, in the film, _The Illusionist_. And like the orange tree, it is a near-complete illusion, a ritual meant to keep the legal department and HR department happy.

The LLMs have simply made the corruption of academia accessible to _all_ students with an internet connection (EDIT: and instantaneous and cheap, unlike a human writer).

There has never been a shortcut to building trust. One cannot LLM their way into being a (metaphorical) druid.

I do not look forward to the Voight-Kampff tests that will come to dominate all aspects of online and asynchronous human interaction.

Note that, short of homework/classwork that _can't_ be gamed by an LLM (for some fundamental reason), even the high-quality honest students will be forced to cheat, so as to not be eclipsed by the actual low-quality cheating students[0].

I imagine that we may end up wrapping around to live in-person dialectics, as were standard in the time of Socrates and Parmenides[1]. If so, this should be fun.

[0]: If left unaddressed, we may see a bimodal distribution of great and terrible students graduating college, with those in between dropping out. If college is an attempt to categorize and rank a population, this would be a major fault in that mechanism.

[1]: Not to the exclusion of the other kinds of tests, writing is still important, critical even. But as a kind of verification-step, that should inform how much the academic community should trust the writing (I can imagine that all the writers here are experiencing stage-fright as they are reading these words).

Re: Training students to prove they're not robots is pushing them to use more AI

#116
post #66

The core of the problem the article is about isn't AI or LLMs, it's about scam software that claims to catch cheating. It's crap for the same reasons that crime predictions software is crap. It's selling a panacea, and that kind of product inherently attracts scammers. If your school uses software to detect AI writing, that's a problem with the quality of your school. The people choosing that software are too stupid…

I'm always startled about how HN approaches these topics. When we have a press release from a university about how researchers can detect thoughts via fMRI, we have no issue with the claim. But if a vendor makes a pretty believable claim that there are repetitive statistical patterns in LLM output, it's all of sudden treated the same as palm reading. The problem isn't that AI detection doesn't work. State of the art…

Eliminating any statistically significant difference between a high-quality human-written text and LLM-written text is exactly what the LLMs are being trained for. At this point, "text is low quality, therefore must be human" is a much stronger signal.

Re: Training students to prove they're not robots is pushing them to use more AI

#117

Tangent: I've noticed I write a lot different because of combative online arguments. I have a problem. So much of my communication is directed to people who don't want to hear me or understand me. So I've become very punchy and repetitive, trying to hammer home ideas that people are either unable or unwilling to understand. I need to find ways to talk to people who want to hear and understand me. It's hard to find ot…

> So much of my communication is directed to people who don't want to hear me or understand me. If they don't want to listen, why waste the time? > So I've become very punchy and repetitive, trying to hammer home ideas that people are either unable or unwilling to understand. If they don't want it, why stuff it down their throats? Aren't they allowed to have their own ideas?

There's a tension (imo) between deciding to only spend time trying to talk to people who immediately agree with you or are open to hearing you out vs those who immediately disagree such that they will fight hard to not hear, not understand, misinterpret, or "not have time for this". The latter is a specific form of disagreement where they've "noise-canceled" the possibility of learning or understanding (even if it would be perfectly reasonable for them to disagree with it afterward).

Is your life easier to not waste time on them? I guess. But obviously you're going to put yourself in a similar bubble, and to whatever extent the issue is important it's now become undiscussed. As you've hinted at, they could be right and you wrong, but the difference is (at least in the premise) that one is willing to talk and listen and so really only one side has the potential to change and it's not based on the merit of the argument—because of course no conversation took place. How hard does one try to encourage someone else to listen? Or rather continuing pursuing a conversation that's being denied? That's the tension. I don't know other than it seems like the side unwilling to listen wins a little bit each time they've successfully evaded it and wins a little more when the other has decided to let it go. I don't just mean they've won a proverbial argument, I mean the issue or decision in question tilts toward their side.

Re: Training students to prove they're not robots is pushing them to use more AI

#118

Perhaps we should not grade students on weekly, or other occasional, writing during the term or semester. How about going back to the old system where, apart from experimental lab work, nothing is graded until the end of the term? All weekly assignments should just be considered prep for one exam at the end of the term where the student has an opportunity to demonstrate mastery of the course's subject matter. They ca…

> one exam at the end of the term where the student has an opportunity to demonstrate mastery of the course's subject matter. The resulting grade depends on that, and that alone. I love this idea. And if a student is having a really bad day, or their dog just died, or they have bad cramps, or they have a hard time dealing with the intense stress of your entire grade being decided in one exam... well, those loser stud…

That's how it was for me - one exam per course at the end of each semester. To qualify for the exam you had to do take-home assignments. Didn't pass? Try again next semester. Was it easy? Hell no, but I learned a lot.

Re: Training students to prove they're not robots is pushing them to use more AI

#119

Earlier quoted context omitted.

If you have spent several days structuring a 20 page argument in October on any topic you'll have learnt a great deal about the subject matter. When you get to the exam hall in, say, May it will stand to you. That knowledge will show up in the blue book vis-a-vis the other exam candidates.

Sure--yes--the student will learn something if they actually wrote a 20-page paper on some given topic. But how are you going to evaluate their ability to compose the 20-page argument ? I would prefer not to be confrontational here, but I am having a hard time imagining that you've deeply considered the pedagogy of how to teach and evaluate students on squishy skills like this. Knowing a bunch of facts about somethin…

In the simplest case, where we'll say the exam question was precisely the topic of the 20 page paper, the candidate would be golden. Of course, it's unlikely in a 3 hr. exam that you'll be asked to write a 20 page response; but in edited form, you could definitely produce three cogent pages about some particular aspect of the original paper - if you've done the work. If you truly wrote the 20 page paper, you can surely produce three literate, cogent, responsive and topical pages.

Re: Training students to prove they're not robots is pushing them to use more AI

#120
post #99
post #84

Earlier quoted context omitted.

This approach does not really solve the core issue. In practice, students often do poorly when evaluation is concentrated in one end of term exam. It also pushes many students to cram at the end of the term instead of learning steadily. A better approach is to rethink what we assess and how we assess it. Research shows that the design of assessments plays an important role in academic integrity. Assignments that requ…

It's a noble motivation (and not even unrealistic in 2015 ) but what you'll get would still be generative output. If the only remedy is monitored end of term exams, so be it.

What exactly is that remediating? I don’t think that approach solves the problem of helping kids learn better.
Post reply on HN