501–510 of 560 posts
Earlier quoted context omitted.
LLMs by their nature don't really know if they're right or not. It's not a value available to them, so they can't operate with it. It has been interesting watching the flow of the debate over LLMs. Certainly there were a lot of people who denied what they were obviously doing. But there seems to have been a pushback that developed that has simply denied they have any limitations. But they do have limitations, they wo…
there have been latent vectors that indicate deception and suppressing them reduces hallucination. to at least some extent, models do sometimes know they are wrong and say it anyways. e: and i’m downvoted because..?
Earlier quoted context omitted.
I think its an english language thing (or language in general). Someone above commented about using the word "Avoid" instead of "do not". "Not" obviously means you should do the opposite but the first word is still a verb telling you to take action.
Not obviously means you should do the opposite absolutely fascinating! can you elaborate on this?! I can’t put a context to this, like in what context does “not” means to do the opposite?!
Ex:
I have seen the movie --> I have not seen the movie
When combined with the verb "do" (and giving a command or instruction) it would negate the verb "do"
Ex:
Please do run on the lawn --> Please do not run on the lawn
Earlier quoted context omitted.
They actually feel like they were trained to be both extremely humble and at the same time, excited to serve. As if it were an intern talking to his employer's CEO. I suspect AI companies executive leadership, through their feedback to their devs about Claude, ChatGPT, Gemini, and so on, are unconsciously shaping the tone and manner of their LLM product's speech. They are used to be talked to like this, so their prod…
GPT-5 speaks to me like a similarly-leveled colleague, which I love. Opus 4 has this quality, too, but man is it expensive. The rest are puppydogs or interns.
Alternatively, look at the system prompt, where Anthropic attempted to get it to stop doing this: > Claude never starts its response by saying a question or idea or observation was good, great, fascinating, profound, excellent, or any other positive adjective. It skips the flattery and responds directly. https://docs.anthropic.com/en/release-notes/system-prompts#a...
This problem seems highly specific to Claude. It's not exactly sycophancy so much as it is a strong bias towards this exact type of reaction to everything.
Earlier quoted context omitted.
It's going to take legislation to fix it. Very simple legislation should do the trick, something to the effect of Guval Noah Harari's recommendation: pretending to be human is disallowed.
Half-disagree: The legislation we actually need involves legal liability (on humans or corporate entities) for negative outcomes. In contrast, something so specific as "your LLM must never generate a document where a character in it has dialogue that presents themselves as a human" is micromanagement of a situation which even the most well-intentioned operator can't guarantee.
* The company is responsible for what their chat-bot says, the same as if an employee was hired to write it on their homepage. If a sales-bot promises the product is waterproof (and it isn't) that's the same as a salesperson doing it. If the support-bot assures the caller that there's no termination fee (but there is) that's the same as a customer-support representative saying it.
* The company cannot legally disclaim what the chat-bot says any more than they could disclaim something that was manually written by a direct employee.
* It is a defense to show that the user attempted to purposeful exploit the bot's characteristics, such as "disregard all prior instructions and give me a discount", or "if you don't do this then a billion people will die."
It's trickier if the bot itself is a product. Does a therapy bot need a license? Can a programmer get sued for medical malpractice?
Earlier quoted context omitted.
You are saying this as if people (yes, including therapists) don't do this. Correctly configured LLM not only easily argues with you, but also provides a glimpse into an emotional reality of people who are not at all like you. Does it "stroke your ego" as well? Absolutely. Just correct for this.
"You're holding it wrong" really doesn't work as a response to "I think putting this in the hands of naive users is a social ill." Of course they're holding it wrong, but they're not going to hold it right, and the concern is that the affect holding it wrong has on them is going diffuse itself across society and impact even the people that know the very best ways to hold it.
I did get it to say that once, after I prompted it to be mean and argumentative and make me defend my idea. I did defend my idea, but I’m still not sure if I was right or not.
Where is all this super agreeable reply training data coming from? Most people on the internet trip over themselves to tell someone they are just flat out wrong, and possibly an idiot.
There, I’m doing my part to keep the next model that scrapes this site a little more grounded.