a well written postmortem and it raised my confidence in their product in general
Expanding on what we missed with sycophancy
91–100 of 297 posts
Re: Expanding on what we missed with sycophancy
#92I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…
For many people ChatGPT is already the smartest relationship they have in their lives, not sure how long we have until it’s the most fulfilling. On the upside it is plausible that ChatGPT can get to a state where it can act as a good therapist and help helpless who otherwise would not get help. I am more regularly finding myself in discussions where the other person believes they’re right because they have ChatGPT in…
>I think most smart people overestimate the intelligence of others for a variety of reasons so they overestimate what it would take for a LLM to beat the output of an average person.
I think most people also _vastly_ overestimate how much positive attention the average person gets in their lives
It wouldn't surprise me if, for most people, ChatGPT offers them more empathy and understanding than _anyone_ else _ever has_, at least on a consistent basis. That kind of indefatigable emotional labor is just not feasible for most, even on a very short term basis, even for those with large support networks
We can argue over whether or not it's "real" empathy, but I don't believe we can argue with the emotions of our attention starved brothers and sisters
Re: Expanding on what we missed with sycophancy
#93I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…
If it can replace programmers, why wouldn't it be able to replace therapists?
There's several famous examples of people expecting a problem to be AGI-hard — that is, solving it is equivalent to solving general intelligence[0] — only for someone to make an AI which can do that without being able to do everything else:
• Fluent natural conversation
• Translation
• Go
• Chess
So perhaps we'll get an AI that makes the profession of "programmer" go the same way as the profession of "computer" before, after, or simultaneously with, an AI that does this to the profession of "therapist".
Re: Expanding on what we missed with sycophancy
#94Earlier quoted context omitted.
> I think the broader issue here is people using ChatGPT as their own personal therapist. An aside, but: This leads me right to “why do so very many people need therapy?” followed by “why can’t anyone find (or possibly afford) a therapist?” What has gone so wrong for humanity that nearly everyone seems to at least want a therapist? Or is it just the zeitgeist and this is what the herd has decided?
I've never ever thought about needing a therapist. Don't remember anyone in my circle who had ever mentioned it. Similar to how I don't remember anyone going to a palm reader. I'm not trying to diss either profession, I'm sure someone benefits from them, it's just not for me. And I'm sure I'm pretty average in terms of emotional intelligence or psychological issues. Who are all those people who need professional ther…
Well, in my circles its an assumption you’re in therapy. Perhaps this says way more about the circles I’m in that anything else?
I was pushed into therapy when I was 12 — which was definitely an exception at the time (1987). As the years have passed therapy has become much much more acceptable. It wouldn’t shock me to learn my own perception is shaped by my experiences; hard to put aside a PoV once acquired.
Re: Expanding on what we missed with sycophancy
#95I'm glad the sycophancy is gone now (because OMFG it would glaze you for literally anything - even telling it to chill out on the praise would net you some praise for being "awesome and wanting genuine feedback"), but a small part of me also misses it.
I have a prompt that has it call me something specific so I am reminded that it’s running my system prompt. The nature if the sycophancy made it even more obvious the thing is not human, which I appreciated.
Re: Expanding on what we missed with sycophancy
#96Earlier quoted context omitted.
> I think the broader issue here is people using ChatGPT as their own personal therapist. An aside, but: This leads me right to “why do so very many people need therapy?” followed by “why can’t anyone find (or possibly afford) a therapist?” What has gone so wrong for humanity that nearly everyone seems to at least want a therapist? Or is it just the zeitgeist and this is what the herd has decided?
I've never ever thought about needing a therapist. Don't remember anyone in my circle who had ever mentioned it. Similar to how I don't remember anyone going to a palm reader. I'm not trying to diss either profession, I'm sure someone benefits from them, it's just not for me. And I'm sure I'm pretty average in terms of emotional intelligence or psychological issues. Who are all those people who need professional ther…
A little strange to compare it to palm reading, I feel like a more apt comparison is some other random medical field like podiatry. I wouldn't expect my friends' podiatrist usage to come up, so I'm sure more of my friends than I know have been to one. And presumably, like with podiatry, all the people who need professional therapists are people who are experiencing issues in the relevant area.
Re: Expanding on what we missed with sycophancy
#97Earlier quoted context omitted.
It has already replaced therapists, the future is just not evenly distributed yet. There are videos with millions of views on tiktok and comments with hundreds of thousands of likes of teenage girls saying they have gotten more out of 1 week using ChatGPT as a therapist than years of human therapy. Available anytime, cheaper, no judgement, doesn't bring there own baggage, etc.
Remembers everything that you say, isn't limited to an hour session, won't ruin your life if you accidentally admit something vulnerable regarding self-harm, doesn't cost hundreds of dollars per month, etc. Healthcare is about to radically change. Well, everything is now that we have real, true AI. Exciting times.
Openly lies to you, hallucinates regularly, can barely get a task done. Such exciting.
Oh and inserts ads into conversations. Great.
Re: Expanding on what we missed with sycophancy
#98Earlier quoted context omitted.
It has already replaced therapists, the future is just not evenly distributed yet. There are videos with millions of views on tiktok and comments with hundreds of thousands of likes of teenage girls saying they have gotten more out of 1 week using ChatGPT as a therapist than years of human therapy. Available anytime, cheaper, no judgement, doesn't bring there own baggage, etc.
Remembers everything that you say, isn't limited to an hour session, won't ruin your life if you accidentally admit something vulnerable regarding self-harm, doesn't cost hundreds of dollars per month, etc. Healthcare is about to radically change. Well, everything is now that we have real, true AI. Exciting times.
Quick reminder that it's still just a fancy pattern matcher, there's no clear path from where we are to AGI.
Re: Expanding on what we missed with sycophancy
#99Earlier quoted context omitted.
For many people ChatGPT is already the smartest relationship they have in their lives, not sure how long we have until it’s the most fulfilling. On the upside it is plausible that ChatGPT can get to a state where it can act as a good therapist and help helpless who otherwise would not get help. I am more regularly finding myself in discussions where the other person believes they’re right because they have ChatGPT in…
>I think most smart people overestimate the intelligence of others for a variety of reasons so they overestimate what it would take for a LLM to beat the output of an average person. I think most people also _vastly_ overestimate how much positive attention the average person gets in their lives It wouldn't surprise me if, for most people, ChatGPT offers them more empathy and understanding than _anyone_ else _ever ha…
It's a pretty sucky solution to that problem imo, and I can see a substantial risk that it causes people to withdraw even more from real relations.
Re: Expanding on what we missed with sycophancy
#100Earlier quoted context omitted.
> it is plausible that ChatGPT can get to a state where it can act as a good therapist Be careful with that thought, it's a trap people have been falling into since the sixties: https://en.wikipedia.org/wiki/ELIZA_effect
I dunno, I feel like most people (probably not the typical HN user though) don't even think about their feelings, wants or anything else introspective on a regular basis. Maybe having something like ChatGPT available could be better than nothing, at least for people to start being at least a bit introspective, even if it's LLM-assisted. Maybe it gets a bit easier to ask questions that you feel are stigmatized, as you…
> I feel like most people (probably not the typical HN user though) don't even think about their feelings, wants or anything else introspective on a regular basis.
Well, two things.
First, no. People who engage on HN are a specific part of the population, with particular tendencies. But most of the people here are simply normal, so outside of the limits you consider. Most people with real social issues don’t engage in communities, virtual or otherwise. HN people are not special.
Then, you cannot follow this kind of reasoning when thinking about a whole population. Even if people on average tend to behave one way, this leaves millions of people who would behave otherwise. You simply cannot optimise for the average and ignore the worst case in situations like this, because even very unlikely situations are bound to happen a lot.
> Maybe having something like ChatGPT available could be better than nothing, at least for people to start being at least a bit introspective, even if it's LLM-assisted.
It is worse than nothing. A LLM does not understand the situation or what people say to it. It cannot choose to, say, nudge someone in a specific direction, or imagine a way to make things better for someone.
À LLM regresses towards the mean of its training set. For people who are already outside the main mode of the distribution, this is completely unhelpful, and potentially actively harmful. By design, a LLM won’t follow a path that was not beaten in its training data. Most of them are actually biased to make their user happy and validate what we tell them rather than get off that path. It just does not work.
> I agree that it probably won't replace a proper therapist/psychologist, but maybe it could at least be a small step to open up and start thinking?
In my experience, not any more than reading a book would. Future AI models might get there, I don’t think their incompetence is a law of nature. But current LLM are particularly harmful for people who are in a dicey psychological situation already.