Earlier quoted context omitted.
Or just: 1 1 2 3 5 8 13 Or: The first president of the united
And that's better? Isn't that just SMS autocomplete?
If that's SMS autocomplete, then chatLLMs are just SMS autocomplete with sugar on top.
Sycophancy is the first LLM "dark pattern"
51–60 of 110 posts
Re: Sycophancy is the first LLM "dark pattern"
#52Tangent: the analysis linked to by the article to another article about rhetorical tricks is pretty interesting. I hadn't realized it consciously, but LLMs really go beyond the em-dashes thing, and part of their tell-tale signs is indeed "punched up paragraphs". Every paragraphs has to be played for maximum effect, contain an opposition of ideas/metaphors, and end with a mic drop!
Some of it is normal in humans, but LLMs do it all the goddamn time, if not told otherwise.
I think it might be for engagement (like the sycophancy) but also because they must have been trained in online conversation, where we humans tend to be more melodramatic and less "normal" in our conversation.
Re: Sycophancy is the first LLM "dark pattern"
#53LLMs get over-analyzed. They’re predictive text models trained to match patterns in their data, statistical algorithms, not brains, not systems with “psychology” in any human sense. Agents, however, are products. They should have clear UX boundaries: show what context they’re using, communicate uncertainty, validate outputs where possible, and expose performance so users can understand when and why they fail. IMO the…
> LLMs get over-analyzed. They’re predictive text models trained to match patterns in their data, statistical algorithms, not brains, not systems with “psychology” in any human sense.
Per the predictive processing theory of mind, human brains are similarly predictive machines. "Psychology" is an emergent property.
I think it's overly dismissive to point to the fundamentals being simple, i.e. that it's a token prediction algorithm, when it's clear to everyone that it's the unexpected emergent properties of LLMs that everyone is interested in.
Re: Sycophancy is the first LLM "dark pattern"
#54"Dark pattern" implies intentionality; that's not a technicality, it's the whole reason we have the term. This article is mostly about how sycophancy is an emergent property of LLMs. It's also 7 months old.
I think at this point it's intentional. They sometimes get it wrong and go too far (breaking suspension of disbelief) but that's the fine-tuning thing. I think they absolutely want people to have a friendly chatbot prone to praising, for engagement.
Re: Sycophancy is the first LLM "dark pattern"
#55Earlier quoted context omitted.
But it IS intentional, more sycophantry usually means more engagement.
Sort of. I'm not sure the consequences of training LLM's based on users' upvoted responses were entirely understood? And at least one release got rolled back.
I think the only thing that's unclear, and what LLM companies want to fine-tune, is how much sycophancy they want. Too much, like the article mentions, and it becomes grotesque and breaks suspension of disbelief. So they want to get it just right, friendly and supportive but not so grotesque people realize it cannot be true.
Re: Sycophancy is the first LLM "dark pattern"
#56[EDIT - Deleted poor humor re how we flatter our pets.]
I am not sure we are going to solve these problems in the time frames in which they will change again, or be moot.
We still haven't brought social media manipulation enabled by vast privacy violating surveillance to heel. It has been 20 years. What will the world look like in 20 more years?
If we can't outlaw scalable, damaging, conflicts of interest (the conflict, not the business), in the age of scaling, how are we going to stop people from finding models that will tell them nice things.
It will be the same privacy violating manipulators who supply sycophantic models. Surveillance + manipulation (ads, politics, ...) + AI + real time. Surveillance informed manipulation is the product/harm/service they are paid for.
Re: Sycophancy is the first LLM "dark pattern"
#57"Dark pattern" implies intentionality; that's not a technicality, it's the whole reason we have the term. This article is mostly about how sycophancy is an emergent property of LLMs. It's also 7 months old.
> "Dark pattern" implies intentionality; that's not a technicality, it's the whole reason we have the term.
The way I think about it is that sycophancy is due to optimizing engagement, which I think is intentional.
Re: Sycophancy is the first LLM "dark pattern"
#58LLMs get over-analyzed. They’re predictive text models trained to match patterns in their data, statistical algorithms, not brains, not systems with “psychology” in any human sense. Agents, however, are products. They should have clear UX boundaries: show what context they’re using, communicate uncertainty, validate outputs where possible, and expose performance so users can understand when and why they fail. IMO the…
> LLMs get over-analyzed. They’re predictive text models trained to match patterns in their data, statistical algorithms, not brains, not systems with “psychology” in any human sense. Per the predictive processing theory of mind, human brains are similarly predictive machines. "Psychology" is an emergent property. I think it's overly dismissive to point to the fundamentals being simple, i.e. that it's a token predict…
The fact that a theory exists does not mean that it is not garbage
Re: Sycophancy is the first LLM "dark pattern"
#59LLMs get over-analyzed. They’re predictive text models trained to match patterns in their data, statistical algorithms, not brains, not systems with “psychology” in any human sense. Agents, however, are products. They should have clear UX boundaries: show what context they’re using, communicate uncertainty, validate outputs where possible, and expose performance so users can understand when and why they fail. IMO the…
they are human in the sense they are reenforced to exhibit human like behavior, by humans. a human byproduct.
Re: Sycophancy is the first LLM "dark pattern"
#60Grok 4.1 thinks my 1-day vibe-coded apps are SOTA-level and rival the most competitive market offerings. Literally tells me they're some of the best codebases it's ever reviewed. It even added itself as the default LLM provider. When I tried Gemini 3 Pro, it very much inserted itself as the supported LLM integration. OpenAI hasn't tried to do that yet.
Grok 4.1 told me my writing surpassed the authors I cited as influence.