Live data from Hacker News

The false positive rate of AI detectors and its effect on freelance writers

authory.com

161–170 of 171 posts

Re: The false positive rate of AI detectors and its effect on freelance writers

#161
post #53

> Desperate, Michael even reached out to the company providing the AI test. To no avail. They simply told him he should set up an account with them, buy himself some credits, and test his articles again himself. At the end of the day, nothing changed the fact that there was this all-encompassing metric being thrown at him saying: You are cheating. Snake Oil Company: "This person's articles were probably -- to almost…

I wish article named the publisher so they can be boycotted.

Re: The false positive rate of AI detectors and its effect on freelance writers

#162
post #98

Earlier quoted context omitted.

I feel the same. I'm not sure there are many organizations out there applying AI detectors to writers they've had an existing, extensive relationship with, and then also giving a fuck if it pops up as a hit but they have no other complaints about the writing. My wife's fairly connected to the writing world and I've not heard of this being a thing at all . Doesn't mean it's not anywhere, but if it were widespread I th…

My experience is that I have to write some things that often have parts that are, for lack of a better word, somewhat boilerplate, e.g. preamble/background/explain some technical term I used/etc. Necessary but not core technical or otherwise differentiated content. I've used LLMs for this as a sort of first draft. I edit the output but it's a perfectly serviceable way to get some words down and save me an hour or two…

In the realm of technical documentation, too, originality isn't a virtue in itself. (And I say this as a technical writer who doesn't even use LLMs regularly.)

If you needed to throw in a sentence or two ahout, e.g., what load balancing is, you're better off lifting a definition from an authoritative source than trying to come up with a "creative" rephrasing for its own sake. Standard terminology is standard because it works.

Re: The false positive rate of AI detectors and its effect on freelance writers

#163
post #46

There's a key sentence hidden right at the end of this article: > They were terrified of Google downranking any AI-generated content, and because of the apparently clear results from their AI checker, they couldn’t be convinced otherwise. It sounds to me like the client might even have believed the evidence that he provided them showing that he didn't use AI to write the work... ... but they're in the SEO game, and t…

This pattern has precedent. I scan releases with a bunch of Anti-Virus software. Not to find any virus, of course, but if one of the scanning softwares "detects" a virus, some customer is also going to "detect" the virus, and we have a problem.

Have you tried Virustotal? Automates pretty much this task.

I’ma Googler, but had to try five search queries to find the name. Obviously didn’t try this in ages, but it used to be straightforward (and list which AV software was used in the scan).

Re: The false positive rate of AI detectors and its effect on freelance writers

#164

I want to see an "AI detector" that just marks everything as being not AI generated. Then, whenever someone is accused of using an AI by some nameless website, they can present their own website that shows they didn't use AI. Professor: "The website said you used an AI." Student: "Well, here's a website claiming I didn't use an AI." Professor: "I don't know if I believe that website, let's look into the merits of how…

I made this one a while back

https://isthiswrittenbyai.surge.sh/

Re: The false positive rate of AI detectors and its effect on freelance writers

#165
post #152

Let's turn around and look at this from the company's point of view, rather than the freelancer's. The company is looking for text that ranks well on Google SEO. That is what the writer is asked to create. The company has cause to fear that anything that looks even remotely AI generated will be flagged and penalized by Google. Google does this regularly, is a black box, and regularly destroys companies when they do i…

Except Google isn't doing what the company thinks it's doing. AI detectors don't work. Using one as a search ranking signal would make search worse by erroneously down-ranking false positives. Google knows this, so it's reasonable to assume they aren't relying on AI detectors. That makes what the company did stupid, especially if they now have to pay to rewrite the content so that it 'beats' that detector's utterly u…

The point in this case is that it doesn’t matter what Google is doing or whether AI detectors work or not. What matters is making the client happy.

Re: The false positive rate of AI detectors and its effect on freelance writers

#166
post #82

Earlier quoted context omitted.

If students what to deprive themselves of the opportunity to learn then that's their perogative.

Yes but this only further waters down the already watered down legitimacy of a degree. I know a lot of people think students should be building things independently of classes but that kind of misses the point of dedicating a full time investment in guided learning at the hands of an expert instructor. If instructors graded rigorously and had a magic cheating detection too (where cheating is defined overly capaciousl…

Easy, don't count coursework towards the degree, all exams.

Re: The false positive rate of AI detectors and its effect on freelance writers

#167
post #152

Earlier quoted context omitted.

Except Google isn't doing what the company thinks it's doing. AI detectors don't work. Using one as a search ranking signal would make search worse by erroneously down-ranking false positives. Google knows this, so it's reasonable to assume they aren't relying on AI detectors. That makes what the company did stupid, especially if they now have to pay to rewrite the content so that it 'beats' that detector's utterly u…

The point in this case is that it doesn’t matter what Google is doing or whether AI detectors work or not. What matters is making the client happy.

Yes, but as someone who has done a lot of freelance writing, I can tell you that 'what makes the client happy' is often orthogonal to 'what's best for the client'. It's challenging to make them see sense when they have a bee in their bonnet about the latest bit of technological or marketing flimflam.

You're often put in the position where your choices are to get fired or do what they want and watch their sales and search rankings tank. Unfortunately, 'I told you so' doesn't seem to cut much ice when that happens.

Re: The false positive rate of AI detectors and its effect on freelance writers

#168
post #156

Earlier quoted context omitted.

Which text did you enter, and into which AI detector? I tried it with my blog, and it was correctly identified by Copy Leaks as written by a human. I don't know how representative https://copyleaks.com/blog/ai-detector-continues-top-accurac... is, but certainly all of them are getting above phrenology detection rates. Even with this guy, the problem wasn't that it misidentified him as AI. It was that "we can't tell"…

""we can't tell" got turned into, "we can't risk" Oh god, we've developed yet another shitty lie detector.

You only just noticed that?

Re: The false positive rate of AI detectors and its effect on freelance writers

#169

In my experience, anyone who can write at a post high school level is accused of being an AI by everyone who can't.

It's interesting to hear about your experience, and it highlights a common phenomenon in the digital age. As technology advances, it's becoming increasingly challenging to distinguish between human and AI-generated content, especially when it comes to writing. The ability of AI models to generate coherent and well-structured text at a post-high school level has indeed improved significantly in recent years. While it'…

I chuckled.

Re: The false positive rate of AI detectors and its effect on freelance writers

#170
post #120
post #77

Earlier quoted context omitted.

It's a certificate of authenticity. You can lie about whether you used AI or not, but your reputation would be demolished if you were found out. You could have proof-of-humanity parties, like key-signing parties. (Partying 'cause you're human seems like a good enough reason.) The reputation would be an asset of yours, a post-monetary currency.

Aah OK, so this is more about signing something in a way that proves you produced it and then saying "I swear I didn't use AI for this, and I'm willing to stake my cryptographic reputation on it".

Yeah. It gets you a Wikipedia effect where you know how to go about getting this information and consumers of art have certain set expectations.
Post reply on HN