261–270 of 560 posts
What llm isn’t a sycophant?
I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.
As Freud said, there is no negation in the unconscious.
And research articles indicate that when the model computes that it should employ sycophantism it becomes less useful in every other way, just like a real sycophant.
Why tech CEOs love LLMs. Ultimate yes man.
Earlier quoted context omitted.
> Versus, if it just gave a curt and unfriendly answer, most people (esp. Americans) I don’t see this as an American thing. It’s an extension of the current Product Management trend to give software quirky and friendly personality. You can see the trend in more than LLM output. It’s in their desktop app that has “Good Morning” and other prominent greetings. Claude Code has quirky status output like “Bamboozling” and…
It most definitely is a American thing - this is why non-native speakers often come out as rude or unfriendly or plain stupid. We don't appreciate how much there is to language.
This can’t possibly be true, can it? Every language must have its own nuance. non native English speakers might not grasp the nuance of English language, but the same could be said for any one speaking another language.
"Perfect question! You've hit the exact technical detail..."
"Excellent question! You've hit on the core technical challenge. You're absolutely right"
"Great technical question!"
Every response have one of these.
I'm pretty sure they want it kissing people's asses because it makes users feel good and therefore more likely to use the LLM more. Versus, if it just gave a curt and unfriendly answer, most people (esp. Americans) wouldn't like to use it as much. Just a hypothesis.
Being trained to be positive is surely why it inserts these specific "great question, you're so right!" remarks, but if you wasn't trained on that, it still couldn't tell you whether you're great or not
> I'm pretty sure they want it kissing people's asses
The American faux friendliness is not what causes the underlying problem here, so all else being equal, they might as well have it kiss your ass. It's what most English speakers expect from a "friendly assistant" after all
[1] https://hn.algolia.com/?dateEnd=1703980800&dateRange=custom&...
Earlier quoted context omitted.
You’re giving them way too much agency. The don’t love anything and cant be malicious. You may get better results by emphasizing what you want and why the result was unsatisfactory rather than just saying “don’t do X” (this principle holds for people as well). Instead of “don’t explain every last detail to the nth degree, don’t explain details unnecessary for the question”, try “start with the essentials and let the…
The idiom “X loves to Y” implies frequency, rather than agency. Would you object to someone saying “It loves to rain in Seattle”? “Malicious compliance” is the act of following instructions in a way that is contrary to the intent. The word malicious is part of the term. Whether a thing is malicious by exercising malicious compliance is tangential to whether it has exercised malicious compliance. That said, I have got…
YMMV on specifics but please consider the possibility that you may benefit from working on promoting and that not all behaviors you see are intrinsic to all LLMs and impossible to address with improved (usually simpler, clearer, shorter) prompts.
I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.
This is a childrearing technique, too: say “please do X”, where X precludes Y, rather than saying “please don’t do Y!”, which just increases the salience, and therefore likelihood, of Y.
I'm starting to think this is a deeper problem with LLMs that will be hard to solve with stylistic changes. If you ask it to never say "you're absolutely right" and always challenge, then it will dutifully obey, and always challenge - even when you are, in fact, right. What you really want is "challenge me when I'm wrong, and tell me I'm right if I am" - which seems to be a lot harder. As another example, one common…
LLMs by their nature don't really know if they're right or not. It's not a value available to them, so they can't operate with it. It has been interesting watching the flow of the debate over LLMs. Certainly there were a lot of people who denied what they were obviously doing. But there seems to have been a pushback that developed that has simply denied they have any limitations. But they do have limitations, they wo…
Neither do humans who have no access to validate what they are saying. Validation doesn't come from the brain, maybe except in math. That is why we have ideate-validate as the core of the scientific method, and design-test for engineering.
"truth" comes where ability to learn meets ability to act and observe. I use "truth" because I don't believe in Truth. Nobody can put that into imperfect abstractions.
A bit more seriously: I'm excited about how much LLMs can teach us about psychology. I'm less excited about the dependency.
---
Adding a bit more substantial comment:
Users of sites like Stack Overflow have reported really disliking answers like "You are solving the wrong problem" or "This is a bad approach".
There are different solutions possible, both for any technical problem, and for any meta-problem.
Whatever garnish you put on top of the problem, the bitter lesson suggests that more data and more problem context improve the solution faster than whatever you are thinking right now. That's why it's called the bitter lesson.