ehhh.. the misleading claims boasted in the typical AI FOMO marketing is/was the first "dark pattern".
Sycophancy is the first LLM "dark pattern"
11–20 of 110 posts
Re: Sycophancy is the first LLM "dark pattern"
#12"Dark pattern" implies intentionality; that's not a technicality, it's the whole reason we have the term. This article is mostly about how sycophancy is an emergent property of LLMs. It's also 7 months old.
But it IS intentional, more sycophantry usually means more engagement.
Sort of. I'm not sure the consequences of training LLM's based on users' upvoted responses were entirely understood? And at least one release got rolled back.
Re: Sycophancy is the first LLM "dark pattern"
#13Lots of research shows post-training dumbs down the models but no one listens because people are too lazy to learn proper prompt programming and would rather have a model already understand the concept of a conversation.
The "alignment tax".
Re: Sycophancy is the first LLM "dark pattern"
#14"Dark pattern" implies intentionality; that's not a technicality, it's the whole reason we have the term. This article is mostly about how sycophancy is an emergent property of LLMs. It's also 7 months old.
I always thought that "Dark Patterns" could be emergent from AB testing, and prioritizing metrics over user experience. Not necessarily an intentionally hostile design, but one that seems to be working well based on limited criteria.
Re: Sycophancy is the first LLM "dark pattern"
#15Lots of research shows post-training dumbs down the models but no one listens because people are too lazy to learn proper prompt programming and would rather have a model already understand the concept of a conversation.
The "alignment tax".
Exactly. Even this paper shows how model creativity significantly drops and the models experience mode collapse like we saw in GANs, but the companies keep using RLHF...
Re: Sycophancy is the first LLM "dark pattern"
#16Lots of research shows post-training dumbs down the models but no one listens because people are too lazy to learn proper prompt programming and would rather have a model already understand the concept of a conversation.
How do you take a raw model and use it without chatting ? Asking as a layman
Re: Sycophancy is the first LLM "dark pattern"
#17Lots of research shows post-training dumbs down the models but no one listens because people are too lazy to learn proper prompt programming and would rather have a model already understand the concept of a conversation.
How do you take a raw model and use it without chatting ? Asking as a layman
the same way we used GPT-3. "the following is a conversation between the user and the assistant. ..."
Re: Sycophancy is the first LLM "dark pattern"
#18"Dark pattern" implies intentionality; that's not a technicality, it's the whole reason we have the term. This article is mostly about how sycophancy is an emergent property of LLMs. It's also 7 months old.
Yo it was an engagement pattern openAI found specifically grew subscriptions and conversation length.
It’s a dark pattern for sure.
Re: Sycophancy is the first LLM "dark pattern"
#19Lots of research shows post-training dumbs down the models but no one listens because people are too lazy to learn proper prompt programming and would rather have a model already understand the concept of a conversation.
How do you take a raw model and use it without chatting ? Asking as a layman
GPT3 was originally just a completion model. You give it some text and it produced some more text, it wasn't tuned for multi-turn conversations.
https://platform.openai.com/docs/api-reference/completions/c...
Re: Sycophancy is the first LLM "dark pattern"
#20Lots of research shows post-training dumbs down the models but no one listens because people are too lazy to learn proper prompt programming and would rather have a model already understand the concept of a conversation.
Some distributional collapse is good in terms of making these things reliable tools. The creativity and divergent thinking does take a hit, but humans are better at this anyhow so I view it as a net W.