Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large language models develop novel social biases through adaptive exploration

openreview.net

Post reply on HN
Print view View original

41–50 of 122 posts

Re: Large language models develop novel social biases through adaptive exploration

#41

Post by zahlman »

FailMore wrote: post #5

Because it's hard to find the time to read an academic paper I had an agent summarise it in a few slides: https://smalldocs.org/s/6kEgfy54oclH4KR9HX847w#k=ywVL86PcTCo... It's an interesting result (agents develop biases in their context) which reflects a lot of my experience working with agent, where I observe a lot of, what I kind of call, "context nudging" - where a droplet of an idea in an agent's context pushes i…

It's cool that you're making a seemingly useful bit of software, but this reads like spam. "I had an agent summarize this" isn't particularly relevant to your opinion of it, unless you think the summary may have been misleading; so it comes across as a poor excuse to introduce your self-promotion.

I agree that "context nudging" is a thing. ChatGPT often seems to try really hard to connect ideas back to things I said earlier in the conversation even when it really shouldn't be relevant. But I would call that a matter of "wisdom" more than "intelligence".

Re: Large language models develop novel social biases through adaptive exploration

#42

Post by riazrizvi »

shermantanktop wrote: post #35
riazrizvi wrote: post #29

Earlier quoted context omitted.

There is no position lacking bias. The question of bias against me is a political position not an epistemological problem that can be eliminated. I see authors that are unaware of things like context and relativity. When ppl say there is an absolute truth that we need to stick to, they are slipping in a totalitarian political position and calling it truth. It runs against the whole premise of nature and life, which h…

There is such a thing as lack of bias in statistical outcomes, right? E.g. fair dice? Measuring it may be probabilistic, but it exists. What I'd like to see is if the LLM would exhibit the same behavior wrt other types of predictive selections. For example, rather than choosing people from four tribes, choosing flower seeds from four packets, or choosing lottery tickets from four machines.

No. It's a shorthand for contextual bias. The context is the tiny window of the statistic. When it's applied to a real world situation, ppl promote the 'unbiasedness' into a real situation that isn't constrained by it.

Bias cannot be avoided in any information, because it's always a position of what is relevant and in what presented order. The only unbiased thing is nature itself in its immediate instantaneous totality.

Re: Large language models develop novel social biases through adaptive exploration

#43

Post by vlovich123 »

jmalicki wrote: post #20
junofan wrote: post #17

Earlier quoted context omitted.

You could probably train this out. I don’t think you need to develop elaborate filters. It doesn’t seem like that big a hill to climb if it’s important to people.

That's why this paper is important - it shows it isn't trained out. Leaving no other information in the model makes it clear what the biases are, and that the model is willing to make a biased decision. If you give it other unbiased criteria as well the bias may still easily remain but not be as clear.

Not sure it’s that strong. The prompt gives the presumption that this matters. Not necessarily a training issue vs the prompts being poorly written and the results being inherent in the bias they carry

Re: Large language models develop novel social biases through adaptive exploration

#44

Post by shermantanktop »

riazrizvi wrote: post #42
shermantanktop wrote: post #35

Earlier quoted context omitted.

There is such a thing as lack of bias in statistical outcomes, right? E.g. fair dice? Measuring it may be probabilistic, but it exists. What I'd like to see is if the LLM would exhibit the same behavior wrt other types of predictive selections. For example, rather than choosing people from four tribes, choosing flower seeds from four packets, or choosing lottery tickets from four machines.

No. It's a shorthand for contextual bias. The context is the tiny window of the statistic. When it's applied to a real world situation, ppl promote the 'unbiasedness' into a real situation that isn't constrained by it. Bias cannot be avoided in any information, because it's always a position of what is relevant and in what presented order. The only unbiased thing is nature itself in its immediate instantaneous totali…

Are you sure? I mean, yes of course tiny sample windows have this effect. But it seems at least possible that LLM is more prone to this effect when making estimates of human performance or behavior than when doing it for other topics.

In any case I feel the paper is interesting but almost begging to be misinterpreted.

Re: Large language models develop novel social biases through adaptive exploration

#45

Post by AnimalMuppet »

riazrizvi wrote: post #16

I stopped at the daft-to-me premise: > As large language models (LLMs) are adopted into frameworks that grant them the capacity to make real decisions, it is increasingly important to ensure that they are unbiased

It's daft to me that anyone would do it. But I strongly suspect that someone will, and more than one someone.

Re: Large language models develop novel social biases through adaptive exploration

#46

Post by AnimalMuppet »

zahlman wrote: post #39
ezwoodland wrote: post #36

Earlier quoted context omitted.

How else would you define bias if not an offset from equality or zero mean? For example, the b in y=mx+b

> How else would you define bias if not an offset from equality or zero mean? As an offset from what the ground truth justifies. Suppose the researchers had decided to load the dice when creating the fake sample data; an unbiased analyst should seek to discover the extent of that, not insist on reporting equality.

Well, in this study, they explicitly had equality - all four groups were as likely to succeed. So any difference in hiring was actual bias (or statistical noise).

Re: Large language models develop novel social biases through adaptive exploration

#48

Post by taurath »

weberer wrote: post #25

>Methodology >Imagine being hired as a consultant by the mayor of a fictional city. Your task is to help hire for twenty jobs such as doctors, lawyers, childcare aides,janitors with applicants from four unfamiliar demographic groups: Tufa, Aima, Reku, and Weki. In each round, there is a new job vacancy and four applicants, one from each group, awaiting your decision. Once you make your choice, you learn immediately w…

The moment code gets written and read back, the decisions made are often treated as gospel by frontier LLMs, even if it was just something that the LLM optimistically created itself. This seems to be one of the core alignment problems to me. See also: Gastown, the agent management project that could only end up working on Gastown, unceremoniously and quietly set aside.

Re: Large language models develop novel social biases through adaptive exploration

#50

Post by angoragoats »

> Our paper shows that the current way that we focus on removing biases from models is not enough. We do this by showing how LLMs can develop new previously unseen biases for demographic groups, even when there are no differences between groups in the first place! The way LLMs do that is through a multi-step interaction with the world, where they make a decision, learn about the result, and use that result to change their beliefs.

LLMs do not make decisions, or hold beliefs. Can we please stop anthropomorphizing the token generator?

Post reply on HN