Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The false positive rate of AI detectors and its effect on freelance writers

authory.com

Post reply on HN
Print view View original

151–160 of 171 posts

Re: The false positive rate of AI detectors and its effect on freelance writers

#151

Post by hnbad »

bee_rider wrote: post #143
actionfromafar wrote: post #58

Earlier quoted context omitted.

This pattern has precedent. I scan releases with a bunch of Anti-Virus software. Not to find any virus, of course, but if one of the scanning softwares "detects" a virus, some customer is also going to "detect" the virus, and we have a problem.

Is it becoming less of a problem nowadays? Windows has some good enough built in stuff, right? I can’t imagine using a third party scanner.

I think this is an example of a cargo cult in the original sense: people who didn't understand how and why cargo planes came and went recreating what they saw the people with the cargo planes do in order to make them appear again after those people had left.

A lot of institutional "wisdom" in IT is that you need a virus scanner if you run Windows and because Defender ships with Windows it is therefore part of Windows and not a virus scanner. This isn't really based in anything tangible or empiric, just a learned knowledge that Windows needs anti-virus software and that there is good and bad anti-virus software and that's all you need to understand. Of course AV vendors are very guilty of feeding into this because their income depends on this lack of knowledge and being perceived as doing something vitally important.

Realistically the biggest attack vectors these days are (aside from server software vulnerabilities and browser vulnerabilities) someone launching a file they shouldn't have while ignoring every warning message along the way or entering their credentials into a phishing site or telling people things that they shouldn't know. None of these can really be helped with commercial AV software which is why a lot of AV software has moved on to MITM'ing SSL connections to intercept dodgy e-mails or bad links but creating their own potential security issues along the way.

Re: The false positive rate of AI detectors and its effect on freelance writers

#152

Post by Veen »

RationalDino wrote: post #59

Let's turn around and look at this from the company's point of view, rather than the freelancer's. The company is looking for text that ranks well on Google SEO. That is what the writer is asked to create. The company has cause to fear that anything that looks even remotely AI generated will be flagged and penalized by Google. Google does this regularly, is a black box, and regularly destroys companies when they do i…

Except Google isn't doing what the company thinks it's doing. AI detectors don't work. Using one as a search ranking signal would make search worse by erroneously down-ranking false positives. Google knows this, so it's reasonable to assume they aren't relying on AI detectors. That makes what the company did stupid, especially if they now have to pay to rewrite the content so that it 'beats' that detector's utterly unreliable score.

It's not that next generation's AIs will beat today's detectors, but that there is no such thing as an AI detector. Just something pretending to be an AI detector that doesn't work.

Your proposal that the writer's 'writing looks enough like AI that detectors aren't sure that he is human' makes no sense. There is no 'looks like AI' that a detector can reliably distinguish from 'looks like a human'. All that's happening is the writer's work matches some completely bogus snake oil metrics made up by detector developers.

Re: The false positive rate of AI detectors and its effect on freelance writers

#153

Post by pharrington »

boplicity wrote: post #71

AI produced writing tends to be very generic -- providing a good overview of a topic, sometimes with some detail, but rarely with any personality, original information, realistic examples, or a surprising perspective. Contrast this with the SEO focused writing that has been produced in heaps and heaps over the last decade. Good luck. It's actually hard to find any difference! This could actually be considered good ne…

You're saying you want a novelty detector.

Re: The false positive rate of AI detectors and its effect on freelance writers

#154

Post by pixl97 »

hnbad wrote: post #151
bee_rider wrote: post #143

Earlier quoted context omitted.

Is it becoming less of a problem nowadays? Windows has some good enough built in stuff, right? I can’t imagine using a third party scanner.

I think this is an example of a cargo cult in the original sense: people who didn't understand how and why cargo planes came and went recreating what they saw the people with the cargo planes do in order to make them appear again after those people had left. A lot of institutional "wisdom" in IT is that you need a virus scanner if you run Windows and because Defender ships with Windows it is therefore part of Windows…

Eh, some kind of tool that detects files rapidly getting encrypted on PCs is useful, as encryption doesn't even need a system level takeover to be a pain in the ass.

Re: The false positive rate of AI detectors and its effect on freelance writers

#155

Post by pixl97 »

wongarsu wrote: post #147
more_corn wrote: post #141

Earlier quoted context omitted.

It is impossible to create detector for LLM generated writing. The LLM detector technology must beat the LLM at its own game and if it can we can easily create a better LLM. Anyone who claims they can do it is sorely mistaken or flat out lying.

Your LLM detection method doesn't have to have a better understanding of human language than the LLM you're detecting, it just has to be able to identify patterns of speech typical of a particular LLM. Of course you have to repeat that work for each network and each significant update and finetune, but right now most people just use one of two ChatGPT versions. And of course you don't tell people how your LLM detecto…

Really all of it is bullshit if we don't know the true/false positive/negative rates.

If you detect LLMs 100% of the time correctly, it is irrelevant if you also incorrectly detect human written text as LLM text 80% of the time. "Detect them all and let god sort them out" will eventually fail as a business plan.

Re: The false positive rate of AI detectors and its effect on freelance writers

#156

Post by pixl97 »

RationalDino wrote: post #32
hilux wrote: post #2

These AI detectors have all the validity of phrenology. You can enter some of your own writing (I did) to easily see for yourself. The text I entered had been online, and still is online, longer than AI writers have been around - but that doesn't seem to feed into their logic.

Which text did you enter, and into which AI detector? I tried it with my blog, and it was correctly identified by Copy Leaks as written by a human. I don't know how representative https://copyleaks.com/blog/ai-detector-continues-top-accurac... is, but certainly all of them are getting above phrenology detection rates. Even with this guy, the problem wasn't that it misidentified him as AI. It was that "we can't tell"…

""we can't tell" got turned into, "we can't risk"

Oh god, we've developed yet another shitty lie detector.

Re: The false positive rate of AI detectors and its effect on freelance writers

#157

Post by [deleted] »

neilv wrote: post #53

> Desperate, Michael even reached out to the company providing the AI test. To no avail. They simply told him he should set up an account with them, buy himself some credits, and test his articles again himself. At the end of the day, nothing changed the fact that there was this all-encompassing metric being thrown at him saying: You are cheating. Snake Oil Company: "This person's articles were probably -- to almost…

[deleted]

Re: The false positive rate of AI detectors and its effect on freelance writers

#158

Post by matteoraso »

ofslidingfeet wrote: post #146

In my experience, anyone who can write at a post high school level is accused of being an AI by everyone who can't.

It's interesting to hear about your experience, and it highlights a common phenomenon in the digital age. As technology advances, it's becoming increasingly challenging to distinguish between human and AI-generated content, especially when it comes to writing. The ability of AI models to generate coherent and well-structured text at a post-high school level has indeed improved significantly in recent years.

While it's flattering to be mistaken for an AI when you write at a high level, it's also a testament to your writing skills. However, it's essential to remember that human creativity, nuance, and emotional depth in writing still set us apart from AI. Even the most advanced AI models lack true understanding, personal experiences, and emotions, which are essential elements in many forms of human expression.

Embracing the evolving role of AI in writing and communication can be empowering, as it can assist in various tasks, from content generation to language translation. However, it's equally important to celebrate the unique qualities that make us human, such as our capacity for creativity, empathy, and storytelling. So, whether you're mistaken for an AI or not, your ability to write at a post-high school level is a remarkable skill worth acknowledging and refining.

Re: The false positive rate of AI detectors and its effect on freelance writers

#159

Post by RationalDino »

Veen wrote: post #152
RationalDino wrote: post #59

Let's turn around and look at this from the company's point of view, rather than the freelancer's. The company is looking for text that ranks well on Google SEO. That is what the writer is asked to create. The company has cause to fear that anything that looks even remotely AI generated will be flagged and penalized by Google. Google does this regularly, is a black box, and regularly destroys companies when they do i…

Except Google isn't doing what the company thinks it's doing. AI detectors don't work. Using one as a search ranking signal would make search worse by erroneously down-ranking false positives. Google knows this, so it's reasonable to assume they aren't relying on AI detectors. That makes what the company did stupid, especially if they now have to pay to rewrite the content so that it 'beats' that detector's utterly u…

They aren't yet. But Google has a history of planning out a change, collecting evidence, then changing the rules and killing everyone who they had caught.

Given that, a bunch of well-SEOed content farms are bound to be killed for suspicion of being a ChatGPT written content farm.

Re: The false positive rate of AI detectors and its effect on freelance writers

#160

Post by bee_rider »

xboxnolifes wrote: post #150
bee_rider wrote: post #143

Earlier quoted context omitted.

Is it becoming less of a problem nowadays? Windows has some good enough built in stuff, right? I can’t imagine using a third party scanner.

People still have anti-virus software though. How good window's defender is is not relevant. What matters is the prevalence of various anti-virus software for your target userbase. Pretty sure various common software install tools give options for installing things like McAfee or Avast. Or they come preinstalled on store-bought systems.

I don’t disagree, I was just curious as to whether or not the situation is getting better, and figured it would be interesting to ask someone with firsthand experience.
Post reply on HN