Live data from Hacker News

AI Detectors Get It Wrong. Writers Are Being Fired Anyway

gizmodo.com

111–120 of 189 posts

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#111

In some cases an AI will make a weird word choice. So do a lot of humans. Sometimes AIs are needlessly wordy. Um...so are a lot of humans. Rinse and repeat. AI detectors are useless. The AIs are training on human writing, so they write fundamentally like humans. How is this not obvious?

In the general internet the reputation of AI writing is that it's writing that's bad/awkward in a way that is often identifiable (by humans) as not having been written by humans. AI detectors are useless, you're right, but for the same reason AI is unreliable in other contexts, not because AI writing is reliably passable.

> in a way that is often identifiable (by humans) as not having been written by humans.

You should check out reddit sometime. It's been nearly twenty years (not hyperbole) of everyone accusing everyone else of being a bot/shill. Humans are utterly incapable of detecting such things. They're not even capable of detecting Nigerian prince emails as scams.

> not because AI writing is reliably passable. "Newspaper editor" used to be a job because human writing isn't reliably passable. I say this not to be glib, but rather because sometimes it's easy for me to forget that. I have to keep reminding myself.

Also, has it not occurred to anyone that deep down in the brainmeat, humans might actually be employing some sort of organic LLM when they engage in writing? That technology actually managed to imitate that faculty at some low level? So even when a human really writes something, it's still an LLM doing so? When you type in the replies to me, are you not trying to figure out what the next word or sentence should be? If you screw it up and rearrange phrases and sentences, are you not doing what the LLM does in some way?

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#112
post #43

Earlier quoted context omitted.

If you're reading nonfiction, it means you're wasting time reading a lot more words when you could have just read the prompt.

Remind you of some entire genres of book? That’s right, business and self-help books! Any of these with an author who’s got actual accomplishments and money before writing the book was almost certainly already ghostwritten from an outline (and so are lots of other books, you’d be surprised, it’s not just these genres). Successful CEOs or people you’ve heard of generally don’t write their own books. Often, they’re ter…

Well, to err is human, to truly screw up you need a computer.

We're going to be blasted to smithereens with LLM-generated "80% should be good enough" garbage.

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#113
I did some research on this in March and developed an opinionated POV, which I'll paste here for anyone interested.

TL;DR: Detecting AI generated content is hard – really hard. The models available today cannot be trusted and should not be used to make important decisions.

In fact, OpenAI took down their detector down last year because they couldn't reach an acceptable level of accuracy:

https://openai.com/blog/new-ai-classifier-for-indicating-ai-...

One open model trained on open data is Hello-SimpleAI's chaptgpt-detector:

https://huggingface.co/Hello-SimpleAI/chatgpt-detector-rober...

https://huggingface.co/datasets/Hello-SimpleAI/HC3

However, that model is not robust and can be tricked by trivial changes:

https://arxiv.org/abs/2307.02599

I verified this result using the playground on hugging face. For example, it is vulnerable to the “one space character” attacks mentioned in the article, severely limiting the usefulness of trying to detect AI content in an adversarial context.

This ridiculous piece of "research" from Forbes has been causing problems:

https://www.forbes.com/sites/technology/article/best-ai-cont...

The Forbes article is credulous and uncritical, beyond mere naiveté and approaching journalistic malpractice, reporting the sales stories and self-reporting benchmarks of self-interested parties as fact. Nevertheless, I've seen several people share it as "insightful" so it's floating around, doing more harm that good IMO.

While all detectors are terrible, Sapling AI has one of the better ones, if only because they are completely open and honest about it's limitations:

https://sapling.ai/docs/api/detector/

https://sapling.ai/ai-content-detector

Sapling AI also wrote an interesting blog post on GPT SIPs (Statistically Improbable Phrases.)

https://sapling.ai/devblog/chatgpt-phrases/

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#114

The one kind of AI detector that could work would be if the AIs store some checksum(s) of each text they write. Then you can ask it if it wrote a certain text. With the naive version of this, you only have to change one word to get around the system. A better version is to checksum smaller segments. Maybe a "chunk size" of 50 words is good. If you find several such chunks in a text, it's pretty clear you have a sligh…

This won't work, as local AI (the one I can run on my budget-friendly laptop without any Internet access) exists today and even beats GPT3.5 in some benchmarks.

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#115
> A few months later, WritersAccess kicked her off the platform anyway. “They said my account was suspended due to excessive use of AI. I couldn’t believe it,” Gasuras said. WritersAccess did not respond to a request for comment.

I think the unfortunate subject of this piece is based in the USA*.

Americans would benefit here from legislation similar to GDPR — it's not only about getting consent to process your personal data, it also gives people the right to contest any automated decision making made solely on an algorithmic basis.

* there is a Kimberly Gasuras who is a freelance writer in the USA, but if you Google me you'll find a director of horror films and at least one other programmer besides myself

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#116
post #83

Earlier quoted context omitted.

Indeed you can losslessly "compress" an LLM's spew into just the prompt (plus any other inputs like values of random variables). But you can also compress a book's entire content into just its ISBN. It's just that books are hopefully more than just statistical mashups of existing content (some books like textbooks and encyclopaedias are kinds of mashup, though one hopes the editors have more than a statistically-base…

You can't regenerate the book from the ISBN. But you can generate the text from the prompt.

Can you, though? I thought LLMs just by virtue of how they work, are non-deterministic. Let alone if new data is added to the LLM, further retraining happens, etc.

Is it possible to get the same output, 1:1, from the same prompt, reliably?

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#118

Anyone who recommends using an AI detector should be the first person fired. No one cares if you use AI or not... judge the fk'ing writing and the quality of the work and stop being a blocker to progress. Same goes for education... and any where else AI touches... fighting calculators and slide rules was a stupid waste of time, and so is fighting AI.

> stop being a blocker to progress

"Says mulching machine maker to tree about to be turned into mulch."

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#119
One popular AI detection package that you can licence with the turnitin academic anti-plagiarism software warns that it may produce false positives if the writing is (1) not by a native English speaker, (2) writing on a technical topic, or (3) neurodiverse.

So yeah ... congrats, you've built a tool to detect autistic Chinese computer scientists!

Re: AI Detectors Get It Wrong. Writers Are Being Fired Anyway

#120

In some cases an AI will make a weird word choice. So do a lot of humans. Sometimes AIs are needlessly wordy. Um...so are a lot of humans. Rinse and repeat. AI detectors are useless. The AIs are training on human writing, so they write fundamentally like humans. How is this not obvious?

A fairly simple and useful AI detector that works uncannily well on student papers: (a) does the text contain "I am an AI" or words to that effect, (b) are there lots of completely made up references?
Post reply on HN