[deleted]
Expanding on what we missed with sycophancy
161–170 of 297 posts
Re: Expanding on what we missed with sycophancy
#162Earlier quoted context omitted.
> no judgement The value of a good therapist is having an empathetic third party to help you make good judgements about your life and learn how to negotiate your needs within a wider social context. Depending on the needs people are trying to get met and how bad the people around them are, a little bit of a self directed chatbot validation session might help them feel less beat down by life and do something genuinely…
Reading between the lines I think a key part of what makes chatbots attractive, re lack of judgment, is they're like talking to a new stranger every session. In both IRL and online discussions sometimes a stranger is the perfect person to talk to about certain things as they have no history with you. In ideal conditions for this they have no greater context about who you are and what you've done which is a very freei…
Also important is simply that the AI is not human.
We all know that however "non-judgmental" another human claims to be, they are having all kinds of private reactions and thoughts that they aren't sharing. And we can't turn off the circuits that want approval and status from other humans (even strangers), so it's basically impossible not to mask and filter to some extent.
Re: Expanding on what we missed with sycophancy
#163This article could have been a sci-fi short story 10 years ago.
Re: Expanding on what we missed with sycophancy
#164Earlier quoted context omitted.
I had a discussion with GPT 4o about the memory system. I'd don't know if any of this is made up but it's a start for further research - Memory in settings is configurable. It is visible and can be edited. - Memory from global chat history is not configurable. Think of it as a system cache. - Both memory systems can be turned off - Chats in Projects do not use the global chat history. They are isolated. - Chats in Pr…
I assume this is being downvoted because I said I ran it by GPT 4o. I don't know how to credit AI without giving the impression that I'm outsourcing my thinking to it
Put simply, GPT has no information about its internals. There is no method for introspection like you might infer from human reasoning abilities.
Expecting anything but an hallucination in this instance is wishful thinking. And in any case, the risk of hallucination more generally means you should really vet information further than an LLM before spreading that information about.
Re: Expanding on what we missed with sycophancy
#165Earlier quoted context omitted.
I stopped using ChatGPT and started using Gemini, both for some coding problems (deep research, amazing to pull out things from docs etc) and for some personal stuff (as a personal therapist as you say), and it is much more honest and frank with me than ChatGPT ever was. I gave it a situation and asked, was I in the wrong, and it told me that I was according to the facts of the case.
Well Google has access to your history of emails and phone contents so it may say more relevant things.
I didn't enable that feature, which is opt-in, and it was still more relevant.
Re: Expanding on what we missed with sycophancy
#166I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…
It creates weird scenarios in other cases too. I asked it to do generate text to speech audio in a wrestler style voice, which ChatGPT doesn’t do.
But… it lied, and produced empty audio clips and weird pictures with text.
Then it:
- said there was a technical problem - said it could not create audio - created weird 1980s computer voice style audio - claimed I was violating a content party.
I said “stop wasting my time” and it spewed a ridiculous apology. I kept asking and it referred me to various websites. I’ve never inadvertently triggered such a wacky hallucination, and I can see how a vulnerable oersom could be troubled by it.
Re: Expanding on what we missed with sycophancy
#167Moments like these make me reevaluate the AI doomer view point. We aren't just toying with access to dangerous ideas (biological weapons, etc) we are toying with human psychology. If something as obvious as harmful sycophancy can slip out so easily, what subtle harms are being introduced. It's like lead in paint (and gasoline) except rewiring our very brains. We won't know the real problems for decades.
There's some pretty foreseeable stuff just considering the existing attention capitalism business model of big tech we all know and loathe. Eventually OpenAI is going to have to make money, and blending ads into answers will be an obvious way. Next step will be maximizing eyeball time on those ads by any means necessary, including all the engagement baiting techniques Meta and other social media companies have already pioneered.
Re: Expanding on what we missed with sycophancy
#168I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…
> But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case Seems like the same issue as the evil vector [1] and it could have been predicted that this would happen. > It's kind of a wild sign of the times to see a tech company issue this kind of post mortem about a flaw in its te…
Well, that's always what LLM-based AI has been. It can be incredibly convincing but the bottom line is it's just flavoring past text patterns, billions of them it's been "trained" on, which is more accurately described as compressed efficiently onto latent space. Like if someone lived for 10,000 years engaging in small talk at the bar, has heard it all, and just kind of mindlessly and intuitively replied with something that sounds plausible for every situation.
Sam Altman is the real sycophant in this situation. GPT is patronizing. Listening to Sam go off on tangent about science fiction scenarios that are just around the corner... I don't know how more people don't see through it.
I kind of get the feeling the people who have to work him every day got sick of his nonsense and just did what he asked for. Targeting the self-help crowd, drive engagement, flatter users, "create the next paradigm of emotionally-enabled humans of perfect agency" or whatever the fuck it he was popping off about to try to motivate the team to compete better with Anthropic.
He clearly isn't very smart. He clearly is product of nepotism. And clearly, LLM "AI" is an overhyped, overwrought version of 20 questions artificial intelligence enabled by mass data scale and NVidia video game graphics. it's been 4 years now of this and AI still tells me the most obviously wrong nonsense every day.
"Are you sure about that?"
"You're absolutely correct to be skeptical of ..."
Re: Expanding on what we missed with sycophancy
#169Earlier quoted context omitted.
It has already replaced therapists, the future is just not evenly distributed yet. There are videos with millions of views on tiktok and comments with hundreds of thousands of likes of teenage girls saying they have gotten more out of 1 week using ChatGPT as a therapist than years of human therapy. Available anytime, cheaper, no judgement, doesn't bring there own baggage, etc.
> no judgement The value of a good therapist is having an empathetic third party to help you make good judgements about your life and learn how to negotiate your needs within a wider social context. Depending on the needs people are trying to get met and how bad the people around them are, a little bit of a self directed chatbot validation session might help them feel less beat down by life and do something genuinely…
The problem with this is they are practicing like medical providers without any quality assurance or controls to ensure they are behaving appropriately.
Therapy is already a bit of grey zone… you can have anyone from a psychologist, a social worker, an untrained deacon, etc “counseling” you. This is worse.
Hell, I’ve been a coach in different settings - players will ask for advice about all sorts of things. There’s a line where you have to say “hey, this is over my head”
Re: Expanding on what we missed with sycophancy
#170Glad this was identified and fixed. I mostly use ChatGPT for learning by checking my assumptions and it was very unhelpful to always be told my statements were incredibly insightful and so great when I want it to challenge them.