Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The false positive rate of AI detectors and its effect on freelance writers

authory.com

Post reply on HN
Print view View original

131–140 of 171 posts

Re: The false positive rate of AI detectors and its effect on freelance writers

#131

Post by simonw »

PumpkinSpice wrote: post #112

There's a lot of people with strong opinions whether these detectors can or cannot work, but I implore you to do the experiment I just did. Go to the top 10 you find in a Google search and paste a sufficiently long sample of your prose. Not a random HN comment - at least 200-400 words of normal, coherent text. It's a game of cat and mouse in the sense that you can build LLMs specifically optimized for evading the cur…

One flaw in this experiment is that most of us here aren't professional writers, which means that text we produce ourselves is probably less likely to trigger a false positive just because we didn't clean up the spelling, grammar and general writing style to the point that it might look like like an LLM wrote it.

Re: The false positive rate of AI detectors and its effect on freelance writers

#132

Post by saghm »

jdiff wrote: post #23
Kim_Bruning wrote: post #16

Earlier quoted context omitted.

What you want is a sort of AI Witches' Scales? https://en.wikipedia.org/wiki/Oudewater#Buildings

Not quite. Oudewater offered fair scales. What this guy's talking about are still rigged scales, just oppositely rigged scales. The only reason nobody was ever found a witch there was because the underlying concepts were BS.

Sure but...since no one actually is a witch (in the sense they were testing for), a fair scale is also one guaranteed to say that they're not a witch.

Re: The false positive rate of AI detectors and its effect on freelance writers

#133

Post by advisedwang »

mezentius wrote: post #56

"This absolute horror scenario is what happened to writer Michael Berben (not his real name)." Absolutely didn't happen. You might occasionally credit a story in the NYT or WSJ using an anonymous source—but when one appears in a blog by a "serial entrepreneur active in the content industry," flogging a subscription product aimed at exactly the sort of marginally-employed writer targeted by the story, I would expect t…

The story is pretty plausible though. I hear students being forced to rewrite genuine original work because of plagiarism detectors, and what happens in schools eventually happens in the workforce.

Re: The false positive rate of AI detectors and its effect on freelance writers

#134

Post by hoosieree »

__loam wrote: post #82
bee_rider wrote: post #63

Earlier quoted context omitted.

Students are in a weird case. Currently instructors aren’t experts in AI tools because, well, they didn’t exist 10 years ago when even the young professors were learning how to do things. The best that can be done at the moment is to teach the old way. The current students are the ones who will need to make the AI based workflows. But obviating intro assignments by having the AI do them doesn’t benefit anybody.

If students what to deprive themselves of the opportunity to learn then that's their perogative.

In an ideal world, "getting an education" and "getting a degree" are the same. In reality, any overlap at all feels like a win. Which is unfortunate, because some of the best learners are terrible students, and vice versa.

Re: The false positive rate of AI detectors and its effect on freelance writers

#135

Post by ElevenLathe »

lemmsjid wrote: post #48
ceejayoz wrote: post #17

Earlier quoted context omitted.

A good example: Put bits of the US Constitution in some AI detectors and they'll tell you it was AI generated. https://arstechnica.com/information-technology/2023/07/why-a...

Interesting--perhaps the AI detectors are right, and this will be how we discover that we are in fact in a simulated universe!

...or at least a simulated democracy.

Re: The false positive rate of AI detectors and its effect on freelance writers

#136

Post by dragonwriter »

lacoolj wrote: post #110
simonw wrote: post #46

There's a key sentence hidden right at the end of this article: > They were terrified of Google downranking any AI-generated content, and because of the apparently clear results from their AI checker, they couldn’t be convinced otherwise. It sounds to me like the client might even have believed the evidence that he provided them showing that he didn't use AI to write the work... ... but they're in the SEO game, and t…

Doesn't this mean Google has an AI-detector that is somehow accurately finding these? Otherwise the majority of its index would be falsely flagged. What do they use?

> Doesn't this mean Google has an AI-detector that is somehow accurately finding these?

No, it means that someone was afraid they would use an AI detector similar to the one that was being used and downrank based on it; nothing in the article suggests Google is in fact using any AI-detector in its ranking algorithm, and I suspect that Google would be quite happy if people used AI (especially Google’s AI) to produce content that otherwise ranked highly on Google algorithms.

Re: The false positive rate of AI detectors and its effect on freelance writers

#137

Post by wwtdtgotiatl »

mezentius wrote: post #56

"This absolute horror scenario is what happened to writer Michael Berben (not his real name)." Absolutely didn't happen. You might occasionally credit a story in the NYT or WSJ using an anonymous source—but when one appears in a blog by a "serial entrepreneur active in the content industry," flogging a subscription product aimed at exactly the sort of marginally-employed writer targeted by the story, I would expect t…

Funny because I've been hearing this almost fortnightly on reddit and freelance forums for a few months now.

And the big middleman platforms siding with the client.

Some people prefer their anonymity, especially when they are put into the public sphere over disputes which can cost them future work. Notice the client wasn't named either.

Re: The false positive rate of AI detectors and its effect on freelance writers

#138

Post by ShamelessC »

simonw wrote: post #46

There's a key sentence hidden right at the end of this article: > They were terrified of Google downranking any AI-generated content, and because of the apparently clear results from their AI checker, they couldn’t be convinced otherwise. It sounds to me like the client might even have believed the evidence that he provided them showing that he didn't use AI to write the work... ... but they're in the SEO game, and t…

> but they're in the SEO game, and they were paranoid about a (so far hypothetical? I don't think Google have announced this yet) AI detection mechanism that Google might use to downrank them.

Google is probably just saying that scare off the SEO spammers from trying to use GPT4 to automate their jobs. I doubt they have a reliable detector (at least not one that detects based solely on the text content in the page).

Re: The false positive rate of AI detectors and its effect on freelance writers

#139

Post by sebzim4500 »

PumpkinSpice wrote: post #112

There's a lot of people with strong opinions whether these detectors can or cannot work, but I implore you to do the experiment I just did. Go to the top 10 you find in a Google search and paste a sufficiently long sample of your prose. Not a random HN comment - at least 200-400 words of normal, coherent text. It's a game of cat and mouse in the sense that you can build LLMs specifically optimized for evading the cur…

I did the same experiment, was pleased to see that all the human-written articles I copied in were correctly identified as human written.

On the other hand, I tried three AI generated texts (>500 words each) and only one was marked as AI generated (it was so obviously AI generated that it would have stood out to me).

Post reply on HN