Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

251–260 of 684 posts

Re: AI overly affirms users asking for personal advice

#251

Post by yard2010 »

alberto467 wrote: post #213
trimbo wrote: post #183

> They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. Sorry, anonymous people on reddit aren't a good comparison. This needs to be studied against people in real life who have a social contract of some sort, because that's what the LLM is imitating, and that's who most people would go to otherwise. Obv…

“AI is nicer than the average redditor” would be a more accurate title

IMHO it's not about being nice. AITA threads show an interesting phenomenon of social consensus, I think the authors wanted to show that the LLMs they checked don't have that.

Re: AI overly affirms users asking for personal advice

#252

Post by layer8 »

post-it wrote: post #149
awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

> I'll check it too by asking "are you just placating me?" the funny thing is that often it'll admit that, yes, it wasn't being very critical, and then procede to over correct and become a complete contrarian. and not in a way that's useful either. It's not admitting anything. Your question diverts it down a path where it acts the part of a former sycophant who is now being critical, because that question is now upst…

I think “admit” here is just a description of what the LLM was saying. It doesn’t imply that the OP thinks the LLM has internal beliefs matching that.

Re: AI overly affirms users asking for personal advice

#253

Post by astrange »

tbrownaw wrote: post #51
sph wrote: post #25

Earlier quoted context omitted.

What do low income people have to do with it, when AI companies and research is borne out of Silicon Valley culture of rich, liberal Californians? I'm still waiting for models based on the curt and abrasive stereotype of Eastern European programmers, as contrast to the sickeningly cheerful AIs we have today that couldn't sound more West Coast if they tried.

> What do low income people have to do with it, when AI companies and research is borne out of Silicon Valley culture of rich, liberal Californians? RLHF is "ask a human to score lots of LLM answers". So the claim is that the AI companies are hiring cheap (~poor) people from convenient locations (CA, since that's where the rest of the company is).

"Poor" in California means earning $80k/year, so they probably are not doing that. Africa / Indonesia / Philippines are better places to find English speaking RLHF workers.

Re: AI overly affirms users asking for personal advice

#254

Post by sublinear »

tbrownaw wrote: post #51
sph wrote: post #25

Earlier quoted context omitted.

What do low income people have to do with it, when AI companies and research is borne out of Silicon Valley culture of rich, liberal Californians? I'm still waiting for models based on the curt and abrasive stereotype of Eastern European programmers, as contrast to the sickeningly cheerful AIs we have today that couldn't sound more West Coast if they tried.

> What do low income people have to do with it, when AI companies and research is borne out of Silicon Valley culture of rich, liberal Californians? RLHF is "ask a human to score lots of LLM answers". So the claim is that the AI companies are hiring cheap (~poor) people from convenient locations (CA, since that's where the rest of the company is).

Yes, this precisely it. There isn't going to be hard evidence to prove it though. Survey data that underpins some empirical studies have similar transparency issues too. This is far from a new problem.

If you adjust your mindset slightly when searching online, it's not hard to find communities of people looking for quick side work and this was huge during the covid lockdown era. There were people helping train LLMs for all kinds of purposes from education to customer service. Those startups quickly cashed out a few years ago and sold to the big players we have now.

I don't get why this is hard for people to believe (or remember)?

Re: AI overly affirms users asking for personal advice

#255

Post by jjmarr »

mattmanser wrote: post #238
alberto467 wrote: post #213

Earlier quoted context omitted.

“AI is nicer than the average redditor” would be a more accurate title

I would say people on /r/amitheasshole are more biased towards the poster, i.e. nicer. There's plenty of those I've read where I thought it sounded like the poster was the asshole and the top replies were NTA.

r/AmItheAsshole is biased towards breaking off relationships rather than fixing them. They also hate social obligations.

e.g. If the OP is asking "I ghosted my friend in AA who insulted me during a relapse", Reddit would say NTA in a heartbeat, while the real world would tell OP to be more forgiving.

On the contrary, if the post was "the other kids at school refuse to play with my child", Reddit would say YTA because the child must've done something to incite being cut off.

Re: AI overly affirms users asking for personal advice

#257

Post by IncreasePosts »

justin_dash wrote: post #38

So at this point I think it's pretty obvious that RLHFing LLMs to follow instructions causes this. I'm interested in a loop of ["criticize this code harshly" -> "now implement those changes" -> open new chat, repeat]: If we could graph objective code quality versus iterations, what would that graph look like? I tried it out a couple of times but ran out of Claude usage. Also, how those results would look like dependi…

In my experience prompting llms to be critical leads then to imagine issues, or to bike shed

Re: AI overly affirms users asking for personal advice

#258

Post by layer8 »

rsynnott wrote: post #119
awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

Why not... do this with a person, instead? Other humans are available. (Seriously, I don't understand this. Plenty of humans will be only too happy to argue with you.)

In addition to availability, usually because you want to take advantage of the knowledge that is baked into the models, which for all its flaws still vastly exceeds the knowledge of any single human.

Re: AI overly affirms users asking for personal advice

#259

Post by tlogan »

This needs to be taken in context. In my view, AI definitely gives better advice than friends, acquaintances, or colleagues (at least in the US culture). But the advice from parents is still the most valuable.

Here is how I would rank it:

1. Parents

2. AI

3. Friends and family

4. Internet search

5. Reddit

Re: AI overly affirms users asking for personal advice

#260

Post by jmkni »

anorwell wrote: post #232

A pastime I have with papers like this is to look for the part in the paper where they say which models they tested. Very often, you find either A) it's a model from one or more years ago, only just being published now, or B) they don't even say which model they are using. Best I could find in this paper: > We evaluated 11 user-facing production LLMs: four proprietary models from OpenAI, Anthropic, and Google; and se…

How many people using AI are actually paying for it (outside of people in tech)?

I find the free models are much more psychophantic and have a higher tendency to hallucinate and just make shit up, and I wonder if these are the ones most people are using?

Post reply on HN