Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

201–210 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#201
post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

The best method for assessing performance when learning is as old as the world: assess the effort, not how well the result complies with some requirements.

If the level of effort made is high, but the outcome does not comply in some way, praise is due. If the outcome complies, but the level of effort is low, there is no reason for praise (what are you praising? mere compliance?) and you must have set a wrong bar.

Not doing this fosters people with mental issues such as rejection anxiety, perfectionism, narcissism, defeatism, etc. If you got good grades at school with little actual effort and the constant praise for that formed your identity, you may be in for a bad time in adulthood.

Teacher’s job is to determine the appropriate bar, estimate the level of effort, and to help shape the effort applied in a way that it improves the skill in question and the more general meta skill of learning.

The issue of judging by the outcome is prevalent in some (or all) school systems, so we can say LLMs are mostly orthogonal to that.

However, even if that issue was addressed, in a number of skills the mere availability of ML-based generative tools makes it impossible to estimate the level of actual effort and to set the appropriate bar, and I do not see how it can be worked around. It’s yet another negative consequence of making the sacred process of producing an amalgamation of other people’s work—something we all do all the time; passing it through the lens of our consciousness is perhaps one of the core activities that make us human—to become available as a service.

Re: Do AI detectors work? Students face false cheating accusations

#203
post #166

Earlier quoted context omitted.

At least the first 2 are far more accurate than humans ever could be. The third, i.e. trusting others to vet and find the correct information, is the problem.

Why? We've done it for ages, most trust in Wikipedia, and before most trusted in encyclopedias. Books written by others have been used forever. We just shift where we place the trust over time.

Agreed, but google hardly gives you those results. Sponsored Ads and AI generated seo crap is hardly an encylopedia.

Re: Do AI detectors work? Students face false cheating accusations

#204

For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy. Text seems to use the same general framework (although words are swapped around) also we see what I call 'word of the week' where whichever 'AI' engine seems to get hung up on a particular English word which is often an unusual one and uses it at every opportunity. It isn't long before you real…

> also we see what I call 'word of the week' where whichever 'AI' engine seems to get hung up on a particular English word which is often an unusual one and uses it at every opportunity

So do humans. Many people have pet phrases or words that they use unusually often compared to others.

Re: Do AI detectors work? Students face false cheating accusations

#205
post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

> I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us.

That basically never happens and the outcome is the result of some sort of struggle. Usually just a peaceful one in the courts and legislatures and markets, but a struggle nonetheless.

> new methods should be used to assess student performance.

Such as? We need an answer now because students are being assessed now.

Return to the old "viva voce" exam? Still used for PhDs. But that doesn't scale at all. Perhaps we're going to have to accept that and aggressively ration higher education by the limited amount of time available for human-to-human evaluations.

Personally I think all this is unpredictable and destabilizing. If the AI advocates are right, which I don't think they are, they're going to eradicate most of the white collar jobs and academic specialties for which those people are being trained and evaluated.

Re: Do AI detectors work? Students face false cheating accusations

#206

Earlier quoted context omitted.

Guns are legal in almost every country - I think your problem is with countries that have almost no restriction on gun ownership. e.g. Here in the UK you can legally own a properly licensed rifle or shotgun and even a handgun in some places outside of Great Britain (e.g. Northern Ireland).

Just because something is technically legal, doesn't mean it's in any way common or part of UK culture to own a gun. There hasn't been a school shooting in the UK for nearly 30 years. Handguns were banned after the last school shooting and there hasn't been one since. https://en.wikipedia.org/wiki/Category:School_shootings_in_t... Although that fact is sometimes forgotten by schools who copy the US in having "active…

Yet the murder rate was unaffected by the gun ban.

Re: Do AI detectors work? Students face false cheating accusations

#207
I'm surprised at the number of comments that give up and say that "AI" is here to stay.

I'm also surprised that academics rely on snake oil software to deal with the issue.

Instead, academics should unite and push for outlawing "AI" or make it difficult to get like cigarettes. Sometimes politicians still listen to academics.

It is probably not going to happen though since the level of political apathy among academics is unprecedented. Everyone is just following orders.

Re: Do AI detectors work? Students face false cheating accusations

#208
post #178

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject. AI is here to stay; new methods should be used to assess student performance. I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with ha…

> I remember being told at school, that we weren't allowed to use calculators in exams I remember being told the same thing, but I happen to believe that it was a fantastic policy, with a lackluster explanation. The idea that you wouldn't have a calculator was obviously silly, even at the time, but underlying observation that relying on the calculator would rob you of the mental exercise the whole ordeal was supposed…

Very well put. I would actually suggest to not use calculators in high school anymore. They add very little value and if it is still the same as when I was in high school, it was a lot of remembering weird key combinations on a TI calculator. Simply make the arithmetic simple enough that a calculator isn't needed.

Re: Do AI detectors work? Students face false cheating accusations

#209
post #166

Earlier quoted context omitted.

At least the first 2 are far more accurate than humans ever could be. The third, i.e. trusting others to vet and find the correct information, is the problem.

Why? We've done it for ages, most trust in Wikipedia, and before most trusted in encyclopedias. Books written by others have been used forever. We just shift where we place the trust over time.

I just googled ‘do I need a license to drive a power boat in UK’

I got AI answer saying ‘no’, but actually you do.

If I use a calculator it will be correct. If I open encyclopaedia it will mostly be correct, because someone with a brain did at least 5 minutes of thining.

We are not talking about some minor detail, AI makes colossal errors with great confidence and conviction.

Re: Do AI detectors work? Students face false cheating accusations

#210

Earlier quoted context omitted.

[flagged]

“Regularly” is not a particularly accurate word. 50 million K12 students in the U.S. — how many mass murders are “regular?”

Any at all? I don't think Americans realize how much of a US-only problem it is, and how some of the non-US mass shootings are explicitly inspired by US media and discourse.
Post reply on HN