Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI overly affirms users asking for personal advice

news.stanford.edu

Post reply on HN
Print view View original

51–60 of 684 posts

Re: AI overly affirms users asking for personal advice

#51

Post by tbrownaw »

sph wrote: post #25
xiphias2 wrote: post #9

Marc Andereseen has talked about the downside of RLHF: it's a specific group of liberal low income people in California who did the rating, so AI has been leaning their culture. I think OpenAI tried to diversify at least the location of the raters somewhat, but it's hard to diversify on every level.

What do low income people have to do with it, when AI companies and research is borne out of Silicon Valley culture of rich, liberal Californians? I'm still waiting for models based on the curt and abrasive stereotype of Eastern European programmers, as contrast to the sickeningly cheerful AIs we have today that couldn't sound more West Coast if they tried.

> What do low income people have to do with it, when AI companies and research is borne out of Silicon Valley culture of rich, liberal Californians?

RLHF is "ask a human to score lots of LLM answers". So the claim is that the AI companies are hiring cheap (~poor) people from convenient locations (CA, since that's where the rest of the company is).

Re: AI overly affirms users asking for personal advice

#52

Post by oldfrenchfries »

neya wrote: post #41

WTF is "yes-men"? Orignal title: AI overly affirms users asking for personal advice Dear mods, can we keep the title neutral please instead of enforcing gender bias?

Thats a fair point on the title. I used "Yes-Men" as a colloquialism for the "sycophancy" described in the Stanford paper, but overly affirming or sycophantic is definitely more precise and neutral. I cant edit the title anymore, but I appreciate the catch.

Re: AI overly affirms users asking for personal advice

#53

Post by cyanydeez »

SlinkyOnStairs wrote: post #42
kibwen wrote: post #36

Earlier quoted context omitted.

> Maybe it's not so sensible to offload the responsibility of clear thinking to AI companies? Markets don't optimize for what is sensible, they optimize for what is profitable.

It's not market driven. AI is ludicrously unprofitable for nearly all involved.

The profit appears to be capturing the political class and it's associated lobbies and monied interests.

Re: AI overly affirms users asking for personal advice

#54

Post by nprateem »

rusty_venture wrote: post #18
nomorewords wrote: post #6

Earlier quoted context omitted.

How is it an indication? I think people on here don't realize that most of the people don't think things through as much as (software) engineers

Wait, other people don’t make decision trees and mind maps and pro/con lists and consult chatbots before making decisions? Are they just flying through life by the seat of their pants? That doesn’t seem like a very solid framework for achieving desired outcomes.

I heard about someone once who could decide whether to buy a new t-shirt in less than 3 months.

Re: AI overly affirms users asking for personal advice

#55

Post by svara »

Yeah, and if you ask it to be critical specifically to get a different perspective or just to avoid this bias, it'll go over the top in the opposite direction.

This is imo currently the top chatbot failure mode. The insidious thing is that it often feels good to read these things. Factual accuracy by contrast has gotten very good.

I think there's a deeper philosophical dimension to this though, in that it relates to alignment.

There are situations where in the grand scheme of things the right thing to do would be for the chatbot to push back hard, be harsh and dismissive. But is it the really aligned with the human then? Which human?

Re: AI overly affirms users asking for personal advice

#58

Post by otabdeveloper4 »

hnfong wrote: post #10
nomorewords wrote: post #6

Earlier quoted context omitted.

How is it an indication? I think people on here don't realize that most of the people don't think things through as much as (software) engineers

In my local(?) community (like in my city, not my industry) there is a saying "if you had to ask for relationship advice, then you probably should break up". There is some rationale to that. People tend to hold onto relationships that don't lead anywhere in fear of "losing" what they "already have". It's probably a comfort zone thing. So if one is desperate enough to ask random strangers online about a relationship,…

> relationships that don't lead anywhere

Relationships are not transactions that are supposed to "lead somewhere".

Re: AI overly affirms users asking for personal advice

#59

Post by cyanydeez »

oldfrenchfries wrote: post #52
neya wrote: post #41

WTF is "yes-men"? Orignal title: AI overly affirms users asking for personal advice Dear mods, can we keep the title neutral please instead of enforcing gender bias?

Thats a fair point on the title. I used "Yes-Men" as a colloquialism for the "sycophancy" described in the Stanford paper, but overly affirming or sycophantic is definitely more precise and neutral. I cant edit the title anymore, but I appreciate the catch.

New title: "LLMs treat you like a Billionaire; you're not"

Re: AI overly affirms users asking for personal advice

#60

Post by magicalhippo »

awithrow wrote: post #34

It feels like I'm fighting uphill battle when it comes to bouncing ideas off of a model. I'll set things up in the context with instructions similar to. "Help me refine my ideas, challenge, push back, and don't just be agreeable." It works for a bit but eventually the conversation creeps back into complacency and syncophancy. I'll check it too by asking "are you just placating me?" the funny thing is that often it'll…

Gemini seems to be fairly good at keeping the custom instructions in mind. In mine I've told it to not assume my ideas are good and provide critique where appropriate. And I find it does that fairly well.
Post reply on HN