Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The false positive rate of AI detectors and its effect on freelance writers

authory.com

Post reply on HN
Print view View original

71–80 of 171 posts

Re: The false positive rate of AI detectors and its effect on freelance writers

#71

Post by boplicity »

AI produced writing tends to be very generic -- providing a good overview of a topic, sometimes with some detail, but rarely with any personality, original information, realistic examples, or a surprising perspective.

Contrast this with the SEO focused writing that has been produced in heaps and heaps over the last decade. Good luck. It's actually hard to find any difference!

This could actually be considered good news. Maybe all of these "AI Detectors" are actually "bad writing detectors." If you're not producing anything original, in terms of real-world examples, style, or substance -- then maybe the writing ought to have been generated by AI anyways.

Re: The false positive rate of AI detectors and its effect on freelance writers

#72

Post by Verdex »

Waterluvian wrote: post #61
CobrastanJorji wrote: post #47

Earlier quoted context omitted.

Be the change you want to see in the world. And when you do, please use a cardboard box with "Very Real Breathalyzer" written in Sharpie on the side and maybe a Nixie tube sticking out the top.

And inside of it is their breathalyzer.

You beat me by four minutes.

I feel this is the only way this has a remote chance of actually working. However, I think you also need to make sure that the judge feels in on the joke as opposed to being the butt of the joke.

Re: The false positive rate of AI detectors and its effect on freelance writers

#73

Post by ghaff »

ekms wrote: post #62

AI writing tends to be bland and repetitive. So if this encourages people to not write like an AI then that's a win

At least the stuff I've generated (and subsequently rewritten) would probably make my spidey senses tingle (though I wouldn't be certain). On the other hand, if I were primarily comparing it to especially low-rent freelance content, I'd probably have a harder time drawing the distinction.

Re: The false positive rate of AI detectors and its effect on freelance writers

#74

Post by __MatrixMan__ »

Kim_Bruning wrote: post #16
Buttons840 wrote: post #9

I want to see an "AI detector" that just marks everything as being not AI generated. Then, whenever someone is accused of using an AI by some nameless website, they can present their own website that shows they didn't use AI. Professor: "The website said you used an AI." Student: "Well, here's a website claiming I didn't use an AI." Professor: "I don't know if I believe that website, let's look into the merits of how…

What you want is a sort of AI Witches' Scales? https://en.wikipedia.org/wiki/Oudewater#Buildings

Presumably the scales have at some point vindicated an actual witch in error.

Re: The false positive rate of AI detectors and its effect on freelance writers

#75

Post by Verdex »

dado3212 wrote: post #4

Alternative reading: company determines that paid contractor is producing content that they can’t distinguish from a borderline free LLM, and decides to use it instead. Isn’t it likely that AI (not the detector) is the real threat to freelance writers’ livelihoods?

And the alternate alternate reading is that the company is afraid of SEO penalties if they can't convince search engines that the content isn't AI and/or potential copyright issues with LLM generated content.

The detector feels a lot like an excuse for whatever they're really concerned about.

Re: The false positive rate of AI detectors and its effect on freelance writers

#76

Post by Terr_ »

simonw wrote: post #50
ganzuul wrote: post #24

Are his articles included in a training set? Are they actually good content or should he try harder to behave like a human? This sounds like a defamation lawsuit. This seems like an issue NFTs could take some pressure of off.

If you're not joking, I would be fascinated to hear how you think NFTs might be relevant to this.

I suspect the parent-poster means "put your fresh writing into a blockchain, so that you can offer that as evidence you wrote it--or at least knew about it--before it showed up later somewhere else."

Re: The false positive rate of AI detectors and its effect on freelance writers

#77

Post by ganzuul »

simonw wrote: post #50
ganzuul wrote: post #24

Are his articles included in a training set? Are they actually good content or should he try harder to behave like a human? This sounds like a defamation lawsuit. This seems like an issue NFTs could take some pressure of off.

If you're not joking, I would be fascinated to hear how you think NFTs might be relevant to this.

It's a certificate of authenticity. You can lie about whether you used AI or not, but your reputation would be demolished if you were found out. You could have proof-of-humanity parties, like key-signing parties. (Partying 'cause you're human seems like a good enough reason.)

The reputation would be an asset of yours, a post-monetary currency.

Re: The false positive rate of AI detectors and its effect on freelance writers

#78

Post by wongarsu »

margalabargala wrote: post #54
kykeonaut wrote: post #40

Earlier quoted context omitted.

Isn't the underlying concept of AI detectors BS as well? You could say something like "We cannot guarantee beyond reasonable doubt that this content is AI generated". Which one could argue is also always true.

This is something that I suspect will get easier in the next couple years as AI research progresses. I could imagine some sort of reversal tool being made such that "given this output and these weights files, how likely is it that these weights produced this output with a reasonable input?" I don't know, that might be incredibly difficult forever, or it might be something that someone has a breakthrough in and manage…

There also seems to be some willingness to have the models watermark their content, and LLMs provide very subtle methods to do so. For example for sufficiently long texts you could bias the token selection to achieve a very specific frequency for certain words; and declare any text where the word "the" makes up between 5.1% and 5.7% of all words, and "which" about 1% of all words to likely be generated by that AI.

One major issue is the feedback loop: There is lots of entrophy to hide a signal in regular English (whether you follow the methods of Stylometry or Steganography), and even an unmodified AI model has a stylometric signature. But if a significant portion of text we consume ends up being written by a handful of AI models, over time people will adopt many of these features. This is obvious right now, with ChatGPT having a certain writing style and some people unconsciously copying that style, and it will remain true no matter how subtle these things become.

Re: The false positive rate of AI detectors and its effect on freelance writers

#79

Post by notShabu »

"better calculators will destroy humanity" seems like a ridiculous statement

but that's what new AI tools that can create good art/music/writing does. They boil down and automate the middle process of calculation.

what was previously "creative" is revealed as calculation from the God/Universe pov and as the range of calculable things grows, the island of im-a-real-human-bc-creativity gets smaller and smaller

defined this way... creativity ends up including some interesting domains like capital allocation and hedge fund management.

these allow for "expressions of freedom" unconstrained by calculation

Re: The false positive rate of AI detectors and its effect on freelance writers

#80

Post by orangechairs »

simonw wrote: post #46

There's a key sentence hidden right at the end of this article: > They were terrified of Google downranking any AI-generated content, and because of the apparently clear results from their AI checker, they couldn’t be convinced otherwise. It sounds to me like the client might even have believed the evidence that he provided them showing that he didn't use AI to write the work... ... but they're in the SEO game, and t…

That sentence caught me, too. The fear of Google downranking AI-generated content with its left hand, while creating and promoting its own Bard bot-generated content with its right hand. Can an ai-generated snake choke on its own tail? Or is it tails...
Post reply on HN