Hello guys, just curious about how can people or systems (computers) detect when a text was written by an LLM. My question is mainly focused to if there is some API or similar to detect if a text was written by an LLM. Thanks!!!
Ask HN: How do systems (or people) detect when a text is written by an LLM
1–10 of 72 posts
Re: Ask HN: How do systems (or people) detect when a text is written by an LLM
#2Humans detect them mostly through pattern matching. However, for systems, my guess is that a ML model is trained on AI genres texts to detect AI generated texts.
Re: Ask HN: How do systems (or people) detect when a text is written by an LLM
#3Pangram is probably the best known example of a detector with low false positives, they have a research paper here: https://arxiv.org/pdf/2402.14873. They do have an API but not sure if you need to request access for it.
For humans I think it just comes down to interacting with LLMs enough to realize their quirks, but that's not really fool-proof.
Re: Ask HN: How do systems (or people) detect when a text is written by an LLM
#4They cannot.
Unfortunately many believe they can, and it is impossible to disprove. So now real people need to write avoiding certain styles, because a lot of other people have decided those are "LLM clues." Bullets, EM Dash, certain common English phases or words (e.g. Delve, Vibrant, Additionally, etc)[0].
Basicaly you need to sprinkle subtle mistakes, or lower the quality of your written communications to avoid accusations that will side-track whatever youre writing into a "you're a witch" argument. Ironically LLM accusations are now a sign of the high quality written word.
[0] https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing
Re: Ask HN: How do systems (or people) detect when a text is written by an LLM
#5I
Re: Ask HN: How do systems (or people) detect when a text is written by an LLM
#6I "detect" them through overuse of some patterns, like "It's not X. It's Y."
This is an artifact of the default LLM writing style, cross-poisoned through training on outputs -- not an "universal" property.
Re: Ask HN: How do systems (or people) detect when a text is written by an LLM
#7People Look For:
Specific language tells, such as: unusual punctuation, including em–dashes and semicolons; hedged, safe statements, but not always; and text that showcases certain words such as “delve”.
Here’s the kicker. If you happen to include any of these words or symbols in your post they’ll stop reading and simply comment “AI slop”. This adds even less to the conversation than the parent, who may well be using an LLM to correct their second or third language and have a valid point to make.
Re: Ask HN: How do systems (or people) detect when a text is written by an LLM
#8They cannot. Unfortunately many believe they can, and it is impossible to disprove. So now real people need to write avoiding certain styles, because a lot of other people have decided those are "LLM clues." Bullets, EM Dash, certain common English phases or words (e.g. Delve, Vibrant, Additionally, etc)[0]. Basicaly you need to sprinkle subtle mistakes, or lower the quality of your written communications to avoid ac…
The key insight is to avoid – em dashes. You’re absolutely right. It’s not the content, it’s the style.
Re: Ask HN: How do systems (or people) detect when a text is written by an LLM
#9They cannot. Unfortunately many believe they can, and it is impossible to disprove. So now real people need to write avoiding certain styles, because a lot of other people have decided those are "LLM clues." Bullets, EM Dash, certain common English phases or words (e.g. Delve, Vibrant, Additionally, etc)[0]. Basicaly you need to sprinkle subtle mistakes, or lower the quality of your written communications to avoid ac…
The key insight is to avoid – em dashes. You’re absolutely right. It’s not the content, it’s the style.
That's an en-dash.
Re: Ask HN: How do systems (or people) detect when a text is written by an LLM
#10They cannot. Unfortunately many believe they can, and it is impossible to disprove. So now real people need to write avoiding certain styles, because a lot of other people have decided those are "LLM clues." Bullets, EM Dash, certain common English phases or words (e.g. Delve, Vibrant, Additionally, etc)[0]. Basicaly you need to sprinkle subtle mistakes, or lower the quality of your written communications to avoid ac…
The key insight is to avoid – em dashes. You’re absolutely right. It’s not the content, it’s the style.
Ironically one of the big tells for me is the "It's not this. It's that."
Your comment uses a comma though so you're probably a real person :)