Expanding on what we missed with sycophancy • Hacker News Board

Re: Expanding on what we missed with sycophancy

#271

Post by bluefirebrand » Sat, May 03, 2025, 3:52 PM UTC

hackable_sand wrote: post #267

bluefirebrand wrote: post #221

Earlier quoted context omitted.

You cannot brute force a problem without understanding the problem I think you have mistaken "understanding the problem" with "knowing a solution to the problem"

You can brute force a problem without understanding the problem.

Would you be kind enough to give an example of this?

I disagree, but I'm open to being convinced

I just don't think you can solve a problem in any way without understanding it first, at least at a high level

I think if you know a solution you want to reach then you can work backwards from it in a brute force way to reach the solution you want

But I don't think you can take a problem where you aren't sure what the solution is and brute force the solution without understanding the problem

Re: Expanding on what we missed with sycophancy

#272

Post by Telemakhos » Sat, May 03, 2025, 3:57 PM UTC

NoboruWataya wrote: post #17

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

Therapy is currently the leading use of AI.

https://hbr.org/2025/04/how-people-are-really-using-gen-ai-i...

Re: Expanding on what we missed with sycophancy

#273

Post by dcrazy » Sat, May 03, 2025, 4:12 PM UTC

Telemakhos wrote: post #272

NoboruWataya wrote: post #17

I found the recent sycophancy a bit annoying when trying to diagnose and solve coding problems. First it would waste time praising your intelligence for asking the question before getting to the answer. But more annoyingly if I asked "I am encountering X issue, could Y be the cause" or "could Y be a solution", the response would nearly always be "yes, exactly, it's Y" even when it wasn't the case. I guess part of the…

Therapy is currently the leading use of AI. https://hbr.org/2025/04/how-people-are-really-using-gen-ai-i...

If this is true, it’s a big problem. A human therapist is bound to a code of ethics and laws. Their patients are in an intentionally vulnerable position—therapy only works when you are completely honest with your therapist and are open to their suggestions. If challenged, a human therapist can explain their reasoning for pursuing a line of questioning, and defend themselves against accusations of manipulation. A large language model can’t do any of that, nor can any of the people who trained it. The LLM can pretend to explain its reasoning, but it has no conviction, no morals, and no fear of consequence. It’s just a black box of statistics.

Re: Expanding on what we missed with sycophancy

#274

Post by malfist » Sat, May 03, 2025, 4:21 PM UTC

labrador wrote: post #175

svat wrote: post #141

Earlier quoted context omitted.

I didn't downvote but it would be because of the "I'd don't know if any of this is made up" — if you said "GPT said this, and I've verified it to be correct", that's valuable information, even it came from a language model. But otherwise (if you didn't verify), there's not much value in the post, it's basically "here is some random plausible text" and plausibly incorrect is worse than nothing.

see my other comments about the trustworthiness about asking a chat system how it's internals work. They have reason to be cagey.

Your personifying a statistical engine. LLMs aren't cagey. They can't be.

Re: Expanding on what we missed with sycophancy

#275

Post by thaumasiotes » Sat, May 03, 2025, 4:25 PM UTC

skydhash wrote: post #211

CouchYam wrote: post #70

Earlier quoted context omitted.

I sometimes prompt the LLM to talk to me as a instructor - to suggest a topic, ask a question, read my response, correct my grammar, and suggest alternate vocabulary where appropriate. This works quite well. Similar to your comment, I am often hesitant to butcher a language in front of a real person :-).

The first step to really learn a language is to be confident and forgive yourself for any mistakes (you’re starting late anyway , and juggling other things). These days, I do my best to learn and reflect. But any mistakes is just a reminder for more learning (and practice).

> forgive yourself for any mistakes (you’re starting late anyway)

Do you think babies don't make mistakes?

Re: Expanding on what we missed with sycophancy

#276

Post by skydhash » Sat, May 03, 2025, 4:28 PM UTC

thaumasiotes wrote: post #275

skydhash wrote: post #211

Earlier quoted context omitted.

The first step to really learn a language is to be confident and forgive yourself for any mistakes (you’re starting late anyway , and juggling other things). These days, I do my best to learn and reflect. But any mistakes is just a reminder for more learning (and practice).

> forgive yourself for any mistakes (you’re starting late anyway) Do you think babies don't make mistakes?

They do, but they also don’t care. Adults tend to do care about that.

Re: Expanding on what we missed with sycophancy

#277

Post by thaumasiotes » Sat, May 03, 2025, 4:32 PM UTC

skydhash wrote: post #276

thaumasiotes wrote: post #275

Earlier quoted context omitted.

> forgive yourself for any mistakes (you’re starting late anyway) Do you think babies don't make mistakes?

They do, but they also don’t care. Adults tend to do care about that.

So what's the idea behind "you're starting late"?

Re: Expanding on what we missed with sycophancy

#278

Post by malfist » Sat, May 03, 2025, 4:42 PM UTC

hiimkeks wrote: post #268

malfist wrote: post #258

Earlier quoted context omitted.

Talk to it and find out. Seriously. I'd say 40% of my conversations end with it suggesting I ask it to find products for me. For example I was trying to determine if I need to screw down stair nosing or if glue and nails were enough and it wanted to help me find screws

Oh yeah, it also did that for me recently when I asked how I would go about designing a PCB that takes M.2 daughterboards. It recommended stems and threads I should buy. Unfortunately all the product links were dead.

I was asking it about lotions for my roseca, and it continually hallucinated products or recommended discontinued ones

Re: Expanding on what we missed with sycophancy

#279

Post by doright » Sat, May 03, 2025, 4:45 PM UTC

drdeca wrote: post #222

danenania wrote: post #156

Earlier quoted context omitted.

We're talking about psychology, therapy, sycophancy, etc. None of this is empirical. If someone thinks they can, say, create a billion dollar startup, whether they can really do it or not is a subjective determination. The AI might tell the person they can do it. You might tell them they can't, that the AI is sycophantic, and that they should stop talking to it because they're losing touch with reality. But is the AI…

That depends on whether they are capable of creating a billion dollar startup. If they aren’t, and I say they aren’t, then I am correct. If they are, and the AI’s output says they are, then AI’s output is correct.

I think it is more complicated than just a matter of being correct or not. Common advice in some creative professions is "don't bother trying to do X for a living, you'll never make it." The point of the advice is not whether or not it is literally correct and the person is better off not bothering - in the general case, it's probably true. The point is that it acts as a filter for those not motivated enough. It's a backhanded sort of test implicitly presented to the aspirant.

Someone who really, really wants to make a billion dollar startup against all odds is going to ignore your advice anyway. In fact, they would ignore any AI's advice on the topic as well. But that kind of arrogance is precisely what's required to be able to pull it off. Someone who quits the moment an AI tells them "don't do it" was not cut out to accomplish such a goal to begin with.

And maybe in the end the startup will only be worth a couple million dollars, but the hubris to believe they could go even further would be what got them that far at all. So "can build a billion dollar startup" ended up being false, but something else was gained in the end.

Re: Expanding on what we missed with sycophancy

#280

Post by taurath » Sat, May 03, 2025, 6:41 PM UTC

cjbgkagh wrote: post #259

taurath wrote: post #241

Earlier quoted context omitted.

Most human intelligence is within a fairly narrow band. Most people I’ve ever met have their own unique intelligences. Perhaps it might be good to meet more people without holding the self-looping dichotomy of “smart people” vs “normal people”. In my experience it tends to lead to huge cognitive errors.

As mentioned I’m in the rather unique position to have analyzed the complete browser history for a substantial number of people - I have learned far more than I wished to. The behaviors of very high IQ people are rather distinct from regular IQ people due to IQ being both largely generic and those genes having other comorbidities. Most obviously is the depression, anxiety, and bipolar disorders. This is so obvious th…

What gets me the most about this is how certain you are that because you have some data about people’s browser history you can make these conclusions. I hope age and experience avails you of these notions - there is no master database that will tell you the nature of people. All you have is the data you have - I’m sure there are many interesting insights and confounding results, but it’s foolish to think you can know anyone from a database, or social media.

Genetic comorbidities with IQ? You at least have a lot to learn about mental health as well if you’re using these as examples.