Expanding on what we missed with sycophancy • Hacker News Board

Re: Expanding on what we missed with sycophancy

#211

Post by skydhash » Sat, May 03, 2025, 4:58 AM UTC

CouchYam wrote: post #70

cjbgkagh wrote: post #64

Earlier quoted context omitted.

I see the same. I'm waiting on LLMs to get good enough that I can use them to help me learn foreign languages - e.g. talk to me about the news in language X. This way I can learn a language in an interesting and interactive way without burdening some poor human with my mistakes. I would build this myself but others will probably beat me too it.

I sometimes prompt the LLM to talk to me as a instructor - to suggest a topic, ask a question, read my response, correct my grammar, and suggest alternate vocabulary where appropriate. This works quite well. Similar to your comment, I am often hesitant to butcher a language in front of a real person :-).

The first step to really learn a language is to be confident and forgive yourself for any mistakes (you’re starting late anyway , and juggling other things).

These days, I do my best to learn and reflect. But any mistakes is just a reminder for more learning (and practice).

Re: Expanding on what we missed with sycophancy

#212

Post by grey-area » Sat, May 03, 2025, 4:58 AM UTC

labrador wrote: post #173

grey-area wrote: post #146

Earlier quoted context omitted.

You are, and you should stop doing that.

Point taken. I admit my comment was silly the way I worded it. Here's the line I’m trying to walk: When I ask ChatGPT about its own internal operations, is it giving me the public info about it's operation, and also possibly revealing propreitary info, or making things up obfuscate and preserve the illusion of authority? Or all three?

Personally I don’t think it has agency so cannot be described as trying to do anything.

It’s predicting what seems most likely as a description given its corpus (and now what you’d like to hear) and giving you that.

The truth is not really something it knows, though it’s very good at giving answers that sound like it knows what it’s talking about. And yes if it doesn’t have an answer from its corpus it’ll just make things up.

Re: Expanding on what we missed with sycophancy

#213

Post by grey-area » Sat, May 03, 2025, 5:01 AM UTC

conception wrote: post #192

getnormality wrote: post #167

Earlier quoted context omitted.

There's some pretty foreseeable stuff just considering the existing attention capitalism business model of big tech we all know and loathe. Eventually OpenAI is going to have to make money, and blending ads into answers will be an obvious way. Next step will be maximizing eyeball time on those ads by any means necessary, including all the engagement baiting techniques Meta and other social media companies have alread…

They have already introduced ads btw. “The "Enshittification" has arrived I asked ChatGPT about the impact of the current tarrifs on inventories over the next few months. It returned a long list of links to toiletries I might want to buy. I asked it why it did that. It replied: "As of April 28, 2025, OpenAl introduced new shopping features to ChatGPT, enhancing its capabilities to provide product recommendations comp…

Are you able to link the chat, I would like to see that.

Re: Expanding on what we missed with sycophancy

#214

Post by drodgers » Sat, May 03, 2025, 5:02 AM UTC

dtech wrote: post #13

labrador wrote: post #11

Earlier quoted context omitted.

They had to after a tweet floated around of a mentally ill person who had expressed psychotic thoughts to the AI. They said they were going off their meds and GPT 4o agreed and encouraged them to do so. Oops.

Are you sure that was real? I thought it was an made up example of the problems with the update

At >500M weekly active users it doesn't actually matter. There will be hundreds of cases like that example that were never shared.

Re: Expanding on what we missed with sycophancy

#215

Post by astrange » Sat, May 03, 2025, 5:06 AM UTC

tomalbrc wrote: post #97

a_wild_dandan wrote: post #90

Earlier quoted context omitted.

Remembers everything that you say, isn't limited to an hour session, won't ruin your life if you accidentally admit something vulnerable regarding self-harm, doesn't cost hundreds of dollars per month, etc. Healthcare is about to radically change. Well, everything is now that we have real, true AI. Exciting times.

Openly lies to you, hallucinates regularly, can barely get a task done. Such exciting. Oh and inserts ads into conversations. Great.

> Oh and inserts ads into conversations. Great.

Are you sure you don't have browser malware?

Re: Expanding on what we missed with sycophancy

#216

Post by istjohn » Sat, May 03, 2025, 5:32 AM UTC

cjbgkagh wrote: post #193

croemer wrote: post #187

Earlier quoted context omitted.

LLMs are already good enough to tell you the news in language X and listen to your broken attempts at asking questions back. Or what's missing?

They do seem close to being able to, I'm mostly waiting on someone to provide it as a service - I don't have time for side projects ATM.

It's not a project. Just open Claude or ChatGPT and tell it what you want to do. You could copy/paste your comment above. If you want to get slightly more fancy, use Google's AIStudio with a simple system prompt.

Re: Expanding on what we missed with sycophancy

#217

Post by istjohn » Sat, May 03, 2025, 5:43 AM UTC

taurath wrote: post #31

cjbgkagh wrote: post #23

Earlier quoted context omitted.

For many people ChatGPT is already the smartest relationship they have in their lives, not sure how long we have until it’s the most fulfilling. On the upside it is plausible that ChatGPT can get to a state where it can act as a good therapist and help helpless who otherwise would not get help. I am more regularly finding myself in discussions where the other person believes they’re right because they have ChatGPT in…

I think that most smart people underestimate the complexity of fields they aren’t in. ChatGPT may be able to replace a psychology listicle, but it has no affect or ability to read, respond, and intervene or redirect like a human can.

There are a lot of awful therapists out there. I wager that Claude Sonnet 3.7 given a suitable, straightforward system prompt would handily outperform non-doctoral degree therapists in a clinical trial, even if the humans had the advantage of in-person sessions.

Re: Expanding on what we missed with sycophancy

#218

Post by istjohn » Sat, May 03, 2025, 5:47 AM UTC

kergonath wrote: post #103

philwelch wrote: post #65

Earlier quoted context omitted.

You’re comparing ChatGPT to an idealized example of a good human therapist when many actual therapists are either useless or even actively harmful to the mental health of their clients.

But then, the fact that harmful therapist exist is not an excuse to make it worse. It’s an excuse to improve regulations. “Car accidents happen regardless of what we do, so YOLO and remove safety standards” is never going to fly.

Displacing those bad therapists would be an improvement.

Re: Expanding on what we missed with sycophancy

#219

Post by [deleted] » Sat, May 03, 2025, 5:48 AM UTC

danenania wrote: post #156

codr7 wrote: post #154

Earlier quoted context omitted.

Reality as in the real world, it is what it is, no one decides.

We're talking about psychology, therapy, sycophancy, etc. None of this is empirical. If someone thinks they can, say, create a billion dollar startup, whether they can really do it or not is a subjective determination. The AI might tell the person they can do it. You might tell them they can't, that the AI is sycophantic, and that they should stop talking to it because they're losing touch with reality. But is the AI…

[deleted]

Re: Expanding on what we missed with sycophancy

#220

Post by Kiro » Sat, May 03, 2025, 5:48 AM UTC

neuroelectron wrote: post #168

photonthug wrote: post #150

Earlier quoted context omitted.

> But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case Seems like the same issue as the evil vector [1] and it could have been predicted that this would happen. > It's kind of a wild sign of the times to see a tech company issue this kind of post mortem about a flaw in its te…

Well, that's always what LLM-based AI has been. It can be incredibly convincing but the bottom line is it's just flavoring past text patterns, billions of them it's been "trained" on, which is more accurately described as compressed efficiently onto latent space. Like if someone lived for 10,000 years engaging in small talk at the bar, has heard it all, and just kind of mindlessly and intuitively replied with somethi…

> I don't know how more people don't see through it.

When you think that maybe you should take a step back and reflect on it. Could it be that your assessment is wrong rather than everyone else being delusional?