Live data from Hacker News

The false positive rate of AI detectors and its effect on freelance writers

authory.com

61–70 of 171 posts

Re: The false positive rate of AI detectors and its effect on freelance writers

#61

Earlier quoted context omitted.

I would kill to watch some smartass show up to court with their own ridiculous-looking black box "breathalyzer" and draw the parallel to "why do you magically trust theirs and not ours?"

Be the change you want to see in the world. And when you do, please use a cardboard box with "Very Real Breathalyzer" written in Sharpie on the side and maybe a Nixie tube sticking out the top.

And inside of it is their breathalyzer.

Re: The false positive rate of AI detectors and its effect on freelance writers

#63

I want to see an "AI detector" that just marks everything as being not AI generated. Then, whenever someone is accused of using an AI by some nameless website, they can present their own website that shows they didn't use AI. Professor: "The website said you used an AI." Student: "Well, here's a website claiming I didn't use an AI." Professor: "I don't know if I believe that website, let's look into the merits of how…

Students are in a weird case. Currently instructors aren’t experts in AI tools because, well, they didn’t exist 10 years ago when even the young professors were learning how to do things. The best that can be done at the moment is to teach the old way. The current students are the ones who will need to make the AI based workflows. But obviating intro assignments by having the AI do them doesn’t benefit anybody.

Re: The false positive rate of AI detectors and its effect on freelance writers

#64

If your work looks so much like generated content that it’s flagging detectors, then it’s also probably the work doomed to be replaced by generated content soon enough (if not already). I don’t think current content generators will destroy many professions any time soon, but a lot of high-volume/low-value work is going to get consumed by them. This seems like an example of that work. Hopefully, this guy didn’t expect…

> If your work looks so much like generated content that it’s flagging detectors, then it’s also probably the work doomed to be replaced by generated content soon enough (if not already).

You assume that the detector actually works as advertised. Without seeing the authors content we have no way to judge in this case. However seeing that even openai struggles with this, i would question that this company has somehow found the holy grail.

In this case if you get wrongly classified you have no way to correct it. This thing might even make your writing worse because now you have to write it in a fashion that does not trigger the detector.

> I don’t think current content generators will destroy many professions any time soon, but a lot of high-volume/low-value work is going to get consumed by them.

In this case you have to compete against the generated AI content and fight against the black box AI detectors as well. You can improve the quality to fight the first. But how do you fight the second? Do i have to get a subscription to every single AI detector on the planet now?

Re: The false positive rate of AI detectors and its effect on freelance writers

#65

Earlier quoted context omitted.

What you want is a sort of AI Witches' Scales? https://en.wikipedia.org/wiki/Oudewater#Buildings

From the linked article: " After the weighing, they received an official certificate proclaiming them not a witch... Certificates would state that 'the body weight is in proportion to its build'. The reasoning behind this is the old belief that a witch has no soul and therefore weighs significantly less than an ordinary person". I love this piece of history so much, thank you for sharing it! Adding this one to my boo…

Also:

“The Waag is still open as a tourist attraction, and official certificates are available.”

Maybe with modern science we can make someone much lighter than they ought to be. (Put helium in their stomach?) Then they would be forced to issue a certificate that said that they could not guarantee that the person is not a witch :D

Re: The false positive rate of AI detectors and its effect on freelance writers

#66

"This absolute horror scenario is what happened to writer Michael Berben (not his real name)." Absolutely didn't happen. You might occasionally credit a story in the NYT or WSJ using an anonymous source—but when one appears in a blog by a "serial entrepreneur active in the content industry," flogging a subscription product aimed at exactly the sort of marginally-employed writer targeted by the story, I would expect t…

I feel the same. I'm not sure there are many organizations out there applying AI detectors to writers they've had an existing, extensive relationship with, and then also giving a fuck if it pops up as a hit but they have no other complaints about the writing.

My wife's fairly connected to the writing world and I've not heard of this being a thing at all. Doesn't mean it's not anywhere, but if it were widespread I think I'd have heard of it. Most companies are trying to push more use of AI tools in writing, as far as I can tell, and I don't even have insight into content-farm parts of the market, where I assume it's just all AI all the time now.

Re: The false positive rate of AI detectors and its effect on freelance writers

#67

Earlier quoted context omitted.

From the linked article: " After the weighing, they received an official certificate proclaiming them not a witch... Certificates would state that 'the body weight is in proportion to its build'. The reasoning behind this is the old belief that a witch has no soul and therefore weighs significantly less than an ordinary person". I love this piece of history so much, thank you for sharing it! Adding this one to my boo…

Which is obviously misguided. You can't determine from somebody's BMI if they have a soul. A soul only weighs three quarters of an ounce [1], or about 1/10th a cup of coffee. Well within even the daily variation of a person's body weight. (/s) 1: https://en.wikipedia.org/wiki/21_grams_experiment

Does my soul look fat? Is it good or bad to have an obese or skinny soul? Do they float? Are they charged, can you trap one in a Faraday cage? Is that what the Ghostbusters were collecting?

Re: The false positive rate of AI detectors and its effect on freelance writers

#68
post #46

There's a key sentence hidden right at the end of this article: > They were terrified of Google downranking any AI-generated content, and because of the apparently clear results from their AI checker, they couldn’t be convinced otherwise. It sounds to me like the client might even have believed the evidence that he provided them showing that he didn't use AI to write the work... ... but they're in the SEO game, and t…

This pattern has precedent. I scan releases with a bunch of Anti-Virus software. Not to find any virus, of course, but if one of the scanning softwares "detects" a virus, some customer is also going to "detect" the virus, and we have a problem.

Yeah. And if you want them to fix their heuristics they tell you to pound sand unless you are willing to pay them a consulting fee to "work with you". Sounds like a protection racket to me, but for some reason it seems to be legal. Will be "great" if we get the same now with "AI detector" companies.

Re: The false positive rate of AI detectors and its effect on freelance writers

#69
post #5

Earlier quoted context omitted.

Kinda like asking for the source code to the breathalyzer? https://arstechnica.com/tech-policy/2007/08/im-not-drunk-off...

I would kill to watch some smartass show up to court with their own ridiculous-looking black box "breathalyzer" and draw the parallel to "why do you magically trust theirs and not ours?"

I feel like the only possible way for this to work would be to get the opposing side to prove that the ridiculous fake breathalyzer doesn't work only for the box to be opened and it is revealed that it is in actuality the same model of breathalyzer that was used against the defendant.

Courts (and authority in general) usually don't appreciate the "oh yeah, well from my point of view the jedi are evil" defense.

However, very occasionally, tricking the offense into skewering their own offensive strategy will work ... that is at least if the deciding authority is in a good mood and you can convince them you're the pragmatic choice.

Post reply on HN