Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Do AI detectors work? Students face false cheating accusations

bloomberg.com

Post reply on HN
Print view View original

171–180 of 1001 posts

Re: Do AI detectors work? Students face false cheating accusations

#171

Post by ndsipa_pomu »

xnorswap wrote: post #150
ndsipa_pomu wrote: post #145

Earlier quoted context omitted.

Guns are legal in almost every country - I think your problem is with countries that have almost no restriction on gun ownership. e.g. Here in the UK you can legally own a properly licensed rifle or shotgun and even a handgun in some places outside of Great Britain (e.g. Northern Ireland).

Just because something is technically legal, doesn't mean it's in any way common or part of UK culture to own a gun. There hasn't been a school shooting in the UK for nearly 30 years. Handguns were banned after the last school shooting and there hasn't been one since. https://en.wikipedia.org/wiki/Category:School_shootings_in_t... Although that fact is sometimes forgotten by schools who copy the US in having "active…

Exactly. It's not the legality of weapons, but the easy availability of them that causes the issues.

It seems to me like victim blaming for U.S. schools to have active shooter drills - it makes more sense to have much better training and screening of gun owners than trying to train the victims. However, given that the NRA is excessively powerful in U.S. politics, I can see why they are necessary, but it just seems easier to me to stop kids from being able to get hold of guns (e.g. have some rudimentary screening for gun purchases and require owners to keep them in locked cabinets when they are not in use).

Re: Do AI detectors work? Students face false cheating accusations

#172

Post by kolinko »

_pdp_ wrote: post #141
woernsn wrote: post #118

Earlier quoted context omitted.

Yes, 3 out of 500 essays were flagged as 100% AI generated. There is a paragraph in the linked article about it.

This study is not very good frankly. Before ChatGPT there was Davinci and other model families which ChatGPT (what became GPT 3.5) was ultimately based on and they are the predecessors of today's most capable models. They should test it on work that is at least 10 to 15 years old to avoid this problem.

What? 10 years ago we wouldn’t dream of what’s happening now.

Models before 2017-2018 (first gpt/bert) didn’t produce any decent text, and before gpt2/gpt3 (2020) you wouldn’t get an essay-grade text.

So you need to go back only 4-5 years to be certain an essay didn’t use AI.

Re: Do AI detectors work? Students face false cheating accusations

#173

Post by kreyenborgi »

Daz1 wrote: post #135
kreyenborgi wrote: post #108

Earlier quoted context omitted.

Obviously false, as LLMs parrot what they're trained on. Not that hard to get them to regurgitate Shakespeare or what have you.

Sounds like a skill issue on your part

Let's test your skills as a plagiarism detector. Below are two paragraphs. One of them was written by an LLM, one by a human. I have only altered whitespace in order to make them scan the same. Can you tell which is which? How much would you bet that you are correct?

A. The Pathfinder and The Deerslayer stand at the head of Cooper's novels as artistic creations. There are others of his works which contain parts as perfect as are to be found in these, and scenes even more thrilling. Not one can be compared with either of them as a finished whole. The defects in both of these tales are comparatively slight. They were pure works of art.

B. The Pathfinder and The Deerslayer stand at the head of Cooper's novels as artistic creations. There are others of his works which contain parts as perfect as are to be found in these, and scenes even more thrilling. Not one can be compared with either of them as a finished whole. The defects in both of these tales are comparatively slight. They were pure works of art.

Re: Do AI detectors work? Students face false cheating accusations

#174

Post by xnorswap »

Nemrod67 wrote: post #169
xnorswap wrote: post #150

Earlier quoted context omitted.

Just because something is technically legal, doesn't mean it's in any way common or part of UK culture to own a gun. There hasn't been a school shooting in the UK for nearly 30 years. Handguns were banned after the last school shooting and there hasn't been one since. https://en.wikipedia.org/wiki/Category:School_shootings_in_t... Although that fact is sometimes forgotten by schools who copy the US in having "active…

let us ban knives then... got a license for that mate?

This is a tired stereotype.

The US has more stabbings per-capita than the UK does, even on top of the shootings.

Re: Do AI detectors work? Students face false cheating accusations

#175

Post by ndsipa_pomu »

Nemrod67 wrote: post #169
xnorswap wrote: post #150

Earlier quoted context omitted.

Just because something is technically legal, doesn't mean it's in any way common or part of UK culture to own a gun. There hasn't been a school shooting in the UK for nearly 30 years. Handguns were banned after the last school shooting and there hasn't been one since. https://en.wikipedia.org/wiki/Category:School_shootings_in_t... Although that fact is sometimes forgotten by schools who copy the US in having "active…

let us ban knives then... got a license for that mate?

There's already a UK ban on carrying knives in public unless you have an occupational need and they're wrapped up or at least not just sitting in your pocket.

Licensing wouldn't be worthwhile as almost every household would want knives for food preparation.

Re: Do AI detectors work? Students face false cheating accusations

#176

Post by crooked-v »

Animats wrote: post #43

Most of the ChatGPT type systems have a rather blah default style. That's what you learn as a non-native speaker of the language. Thus the problem for people who learned English from textbooks. Amusingly, you can push ChatGPT type systems into other styles of writing. I put in the preamble to the US constitution and asked for different styles: Modern: We, the people of the United States, come together to build a stro…

On that note, quite a lot of "AI speak" quirks are actually just the normal writing style of non-Western English speaking cultures, such as the use of "delve" in Nigeria (https://www.theguardian.com/technology/2024/apr/16/techscape...).

Re: Do AI detectors work? Students face false cheating accusations

#177

Post by aleph_minus_one »

greatartiste wrote: post #111

For a human who deals with student work or reads job applications spotting AI generated work quickly becomes trivially easy. Text seems to use the same general framework (although words are swapped around) also we see what I call 'word of the week' where whichever 'AI' engine seems to get hung up on a particular English word which is often an unusual one and uses it at every opportunity. It isn't long before you real…

> The staff soon realised what garbage these tools are so if a student accused of plagiarism decided to argue back then the accusation would be quietly dropped.

I ask myself when the time comes that some student will accuse the stuff of libel or slander becuase of false AI plagiarism accusations.

Re: Do AI detectors work? Students face false cheating accusations

#178

Post by lwhi »

It is no longer effective to solely use a written essay to measure how deeply a student comprehends a subject.

AI is here to stay; new methods should be used to assess student performance.

I remember being told at school, that we weren't allowed to use calculators in exams. The line provided by teachers was that we could never rely on having a calculator when we need it most—obviously there's irony associated with having 'calculators' in our pockets 24/7 now.

We need to accept that the world has changed; I only hope that we get to decide how society responds to that change together .. rather than have it forced upon us.

Re: Do AI detectors work? Students face false cheating accusations

#179

Post by DoingIsLearning »

jfengel wrote: post #160
jillesvangurp wrote: post #155

I'd expect smart people to be able to use tools to make their work easier. Including AI. The bigger picture here is that the current generation of students are going to be using and relying on AI the rest of their careers anyway. Making them do things the old fashioned way is not a productive way to educate them. The availability of these tools is actually an opportunity to raise the ambition level quite a bit. Unive…

We may well need to invent new mechanisms for teaching, but I don't expect that to appear overnight. The point of essays is not to have essays written. The teacher already knows. The point is to practice putting together a coherent thought. The process, not the product, is a the goal. Eventually we'll come up with a way to demonstrate that along with, rather than despite, AI. But for the moment we have machines that…

> We may well need to invent new mechanisms for teaching,

For additional context the short essay format as an evaluation tool is very much a Anglo-saxon university form factor.

There are several other cultures in the world, in particular stemming from Latin/Francophone school of thought, in the old 'cathedra' style university where students are either subjected to written exams only or even historically (less so nowadays) also 'oral' exams (Oratory not dental exams).

Post reply on HN