Live data from Hacker News

Expanding on what we missed with sycophancy

openai.com

81–90 of 297 posts

Re: Expanding on what we missed with sycophancy

#81

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

If it can replace programmers, why wouldn't it be able to replace therapists?

Re: Expanding on what we missed with sycophancy

#82

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

Employees from OpenAI encouraged people to use ChatGPT as their therapist, so yeah, they now have to take responsibility for it

Re: Expanding on what we missed with sycophancy

#83

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

> I think the broader issue here is people using ChatGPT as their own personal therapist. An aside, but: This leads me right to “why do so very many people need therapy?” followed by “why can’t anyone find (or possibly afford) a therapist?” What has gone so wrong for humanity that nearly everyone seems to at least want a therapist? Or is it just the zeitgeist and this is what the herd has decided?

It's a modern variant on Heller's Catch-22: You have to be CRAZY to not want a therapist.

Re: Expanding on what we missed with sycophancy

#84

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

It has already replaced therapists, the future is just not evenly distributed yet. There are videos with millions of views on tiktok and comments with hundreds of thousands of likes of teenage girls saying they have gotten more out of 1 week using ChatGPT as a therapist than years of human therapy. Available anytime, cheaper, no judgement, doesn't bring there own baggage, etc.

Re: Expanding on what we missed with sycophancy

#86
> the update introduced an additional reward signal based on user feedback—thumbs-up and thumbs-down data from ChatGPT. This signal is often useful; a thumbs-down usually means something went wrong.

> We also made communication errors. Because we expected this to be a fairly subtle update, we didn't proactively announce it.

that doesn't sound like a "subtle" update to me. also, why is "subtle" the metric here? i'm not even sure what it means in this context.

Re: Expanding on what we missed with sycophancy

#87

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

It has already replaced therapists, the future is just not evenly distributed yet. There are videos with millions of views on tiktok and comments with hundreds of thousands of likes of teenage girls saying they have gotten more out of 1 week using ChatGPT as a therapist than years of human therapy. Available anytime, cheaper, no judgement, doesn't bring there own baggage, etc.

Is a system optimised (via RLHF) for making people feel better in the moment, necessarily better at the time-scale of days and weeks?

Re: Expanding on what we missed with sycophancy

#88
post #55

Earlier quoted context omitted.

Do you presume that "what people do" is "what they should do"?

If you are suggesting that people shouldn't underestimate the difficulty of the jobs of others - my answer is a strong yes. People should strive for accuracy in all cases. But I did suggest that even if true it does not negate my assertion so I am failing to see the relevance. Perhaps I have misunderstood your point.

Sorry, I was rather obscure - you said "My estimations don’t come from my assumption that other people’s jobs are easy, they come from doing applied research in behavioral analytics on mountains of data in rather large data centers."

And so I considered the preceding discussion in light of your last sentence. Which makes it sound like you are saying "I've observed the behavior of people and they're often flawed and foolish, regardless of the high ideals they claim to be striving for and the education they think they have. Therefore, they will do better with ChatGPT as a companion than with a real human being". But that's quite a few words that you may not have intended, for which I apologize!

What did you mean?

Re: Expanding on what we missed with sycophancy

#89

Earlier quoted context omitted.

I see the same. I'm waiting on LLMs to get good enough that I can use them to help me learn foreign languages - e.g. talk to me about the news in language X. This way I can learn a language in an interesting and interactive way without burdening some poor human with my mistakes. I would build this myself but others will probably beat me too it.

I sometimes prompt the LLM to talk to me as a instructor - to suggest a topic, ask a question, read my response, correct my grammar, and suggest alternate vocabulary where appropriate. This works quite well. Similar to your comment, I am often hesitant to butcher a language in front of a real person :-).

The problem is, AI doesn't let you, or encourage you to create your own style. Word choices, structure, flow, argument building and discourse style is very fixed and "average", since it's a machine favors what it ingests most.

I use Grammarly for grammar and punctuation, and disable all style recommendations. If I let it loose on my piece of text, it converts it to a slop. Same bland, overly optimistic toned text generator output.

So, that machine has no brain, use your own first.

Re: Expanding on what we missed with sycophancy

#90

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

It has already replaced therapists, the future is just not evenly distributed yet. There are videos with millions of views on tiktok and comments with hundreds of thousands of likes of teenage girls saying they have gotten more out of 1 week using ChatGPT as a therapist than years of human therapy. Available anytime, cheaper, no judgement, doesn't bring there own baggage, etc.

Remembers everything that you say, isn't limited to an hour session, won't ruin your life if you accidentally admit something vulnerable regarding self-harm, doesn't cost hundreds of dollars per month, etc.

Healthcare is about to radically change. Well, everything is now that we have real, true AI. Exciting times.

Post reply on HN