Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do systems (or people) detect when a text is written by an LLM

news.ycombinator.com

Post reply on HN
Print view View original

51–60 of 72 posts

Re: Ask HN: How do systems (or people) detect when a text is written by an LLM

#51

Post by fleebee »

alex43578 wrote: post #11
Someone1234 wrote: post #4

They cannot. Unfortunately many believe they can, and it is impossible to disprove. So now real people need to write avoiding certain styles, because a lot of other people have decided those are "LLM clues." Bullets, EM Dash, certain common English phases or words (e.g. Delve, Vibrant, Additionally, etc)[0]. Basicaly you need to sprinkle subtle mistakes, or lower the quality of your written communications to avoid ac…

Someone with native fluency in American English can (should) be able to tell the difference between human writing and unpolished AI copy-paste. Essentially 0 people use emoji to create a bulleted list. Nobody unintentionally cites fake legal precedents or non-existent events, articles, or papers. Even the “it’s not X, it’s Y” structure, in the presence of other suspicious style/tone cues signals LLM text.

I think the trope in this comment[0] from another thread is the most obvious tell, perhaps even more than "not x, but y".

> It’s the fake drama. Punchy sentences. Contrast. And then? A banal payoff.

It's great because it's a double-decker of annoying marketing copy style and nonsensical content.

[0]: https://news.ycombinator.com/item?id=47615075

Re: Ask HN: How do systems (or people) detect when a text is written by an LLM

#52

Post by gwbas1c »

I don't think you can 100% detect AI content, because at some point someone will just prompt the AI to not sound like AI.

I think the better question to ask is: What are your goals? Is it to prevent AI SPAM, or to discourage people copy-pasting AI? Those are two very different problems: in the case of AI SPAM you look for patterns of usage, (IE, unusually high interaction from a single IP, timing patterns around when things are read and the response comes in,) and in the other case it all comes down to cultural norms.

Re: Ask HN: How do systems (or people) detect when a text is written by an LLM

#55

Post by noufalibrahim »

It's a lot easier to detect when you mostly interact with non English speakers.

I asked an LLM to rewrite this to make it nicer and got the following. I'd flag the first because I don't usually hear "majority of your interactions" in conversation but I might miss it. The second will probably get by me. As for the third, I never say "considerably easier" unless I'm trying to sound artificially posh.

1. It becomes much more noticeable when the majority of your interactions are with non-native English speakers.

2.It tends to stand out more when most of the people you interact with speak English as a second language.

3. It's considerably easier to identify when most of your interactions involve people whose primary language isn't English.

Re: Ask HN: How do systems (or people) detect when a text is written by an LLM

#56

Post by fwip »

Filligree wrote: post #46
alex43578 wrote: post #40

Earlier quoted context omitted.

One of my subtle favorites is the “H2 Heading with: Colorful Description” Eg - The Strait of Hormuz: Chokepoint or Opportunity?

I’ve used titles like that for thirty years.

Sure, and an LLM-written article will use that pattern eight times in two pages.

Re: Ask HN: How do systems (or people) detect when a text is written by an LLM

#57

Post by sigotirandolas »

I don't look at whether the text is written by an LLM but at whether it has substance and whether the writer understands what they are doing and is respecting my time.

If the text is full of punchy three word phrases or nonsense GenAI images then that's an obvious sign. But so is if the other person has some revolutionary project with great results but they can't really explain why their solution works where presumably many failed in the past (or it's a word salad, or some lengthy writing that doesn't show any signs of getting you to an "aha, that's some great insight" moment).

A good sign is also if the author had something interesting going before 2022, and they didn't fall into the earliest low quality LLM waves. Unfortunately some genuinely talented people have started using LLMs to turbocharge their output while leaving some quality on the table nowadays, so I don't really know. I'm becoming a lot more sceptical of the Internet, to be honest.

Re: Ask HN: How do systems (or people) detect when a text is written by an LLM

#60

Post by order-matters »

lelanthran wrote: post #48
Someone1234 wrote: post #4

They cannot. Unfortunately many believe they can, and it is impossible to disprove. So now real people need to write avoiding certain styles, because a lot of other people have decided those are "LLM clues." Bullets, EM Dash, certain common English phases or words (e.g. Delve, Vibrant, Additionally, etc)[0]. Basicaly you need to sprinkle subtle mistakes, or lower the quality of your written communications to avoid ac…

> Ironically LLM accusations are now a sign of the high quality written word. Citation needed. The LLM accusations come from the specific cadence they use. You can remove all em-dashes from a piece of text and it still becomes clear when something is LLM written. Can they be prompted to be less obvious? Sure, but hardly anyone does that. It's more "The Core Insight", "The Key Takeaway", etc. than it is about emdashes…

i think another part of the problem is that some people are using AI so much that they are starting to mimic its cadence in their own writing. they may have had a prior coincidental predisposition for writing somewhat similar to AI with worse grammar, and now are inching towards alignment as they either intentionally or accidentally use AI output as a model to improve their writing
Post reply on HN