151–160 of 183 posts
Perhaps we should not grade students on weekly, or other occasional, writing during the term or semester. How about going back to the old system where, apart from experimental lab work, nothing is graded until the end of the term? All weekly assignments should just be considered prep for one exam at the end of the term where the student has an opportunity to demonstrate mastery of the course's subject matter. They ca…
> The assignment had been to write an essay about Kurt Vonnegut’s Harrison Bergeron—a story about a dystopian society that enforces “equality” by handicapping anyone who excel Did not this self censorship process started decades ago? There are certain answers expected in academia, arguing for anything else would get you in troubles. Not using “devoid” seems pretty minor inconvenience. For me biggest wtf is why studen…
Avoid the theory-heavy disciplines. You won't be told what to think (as often) if you take History and Geography rather than Sociology and Gender Studies.
Earlier quoted context omitted.
You can detect if texts from a year ago used AI based on statistical patterns. Nobody is taking issue with that. But once you tell people "we will run these tests to detect if your future submissions are using AI" you create an adversarial environment and your statistical methods will continuously break. Not because statistics is broken, but because you are trying to hit a moving target that doesn't want to be hit. T…
> You can detect if texts from a year ago used AI based on statistical patterns. I don't understand your argument. The vendors for these detection tools can acquire recent samples from all frontier models just as easily as you can use them to write essays. There's nothing that requires a one-year delay.
Earlier quoted context omitted.
You can detect if texts from a year ago used AI based on statistical patterns. Nobody is taking issue with that. But once you tell people "we will run these tests to detect if your future submissions are using AI" you create an adversarial environment and your statistical methods will continuously break. Not because statistics is broken, but because you are trying to hit a moving target that doesn't want to be hit. T…
> you create an adversarial environment Do AI vendors specifically train models to circumvent AI detectors? Why would they?
Earlier quoted context omitted.
>> But if a vendor makes a pretty believable claim that there are repetitive statistical patterns in LLM output, it's all of sudden treated the same as palm reading. That's what fortunetellers do. The problem isn't guessing correctly about AI content in writing. The problem is false positives. That's what puts it in the same category is predictive policing scam software. And fortunetelling.
It has nothing to do with predictive policing. I don't understand this example, it has nothing to do with detecting intent. You're looking for evidence of a past misdeed. False positive and false negative rates are non-zero, as with almost anything, but the tools are pretty good. I encourage you to give them a try. Pangram is a good state-of-the-art choice and you can try it for free. They also publish evals and othe…
I have given them a try and can confirm the exact opposite. Plenty of others have given them tries and have confirmed the exact opposite.
Regardless, the “better for a hundred guilty men to go free than for one innocent man to hang” principle applies here.
The core of the problem the article is about isn't AI or LLMs, it's about scam software that claims to catch cheating. It's crap for the same reasons that crime predictions software is crap. It's selling a panacea, and that kind of product inherently attracts scammers. If your school uses software to detect AI writing, that's a problem with the quality of your school. The people choosing that software are too stupid…
Does crapping on the average school's deep well of expertise for evaluating how effectively AI software solutions address their problems somehow fix the underlying problem (that the cost of catching cheaters is significantly higher than the cost of cheating)? (This is roughly the same problem as evaluating software that only does an approximation of what it claims to do.) (Aside: AI-based variations on this theme are…
This is tackling the problem from the wrong direction. The right direction would be to make it harder to cheat in the first place. For example: if the student submits an essay, and that student is able to coherently and accurately answer any questions asked about the essay in a face-to-face conversation, then that student is probably the genuine author of that essay.
[flagged]
Earlier quoted context omitted.
Right? It’s the obvious question: why are they using software to detect software? I can detect AI written prose in less than five seconds; I would expect a trained teacher to be able to do that as well.
You know you can't just say "I detect AI written prose" and then do whatever you want about it, right? It's not difficult, sure, to detect it. It's difficult to prove that it's true and then punish the student for it.