Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

People are just as bad as my LLMs

wilsoniumite.com

Post reply on HN
Print view View original

11–20 of 173 posts

Re: People are just as bad as my LLMs

#11

Post by d4mi3n »

aidos wrote: post #8
tehsauce wrote: post #3

There has been some good research published on this topic of how RLHF, ie aligning to human preferences easily introduces mode collapse and bias into models. For example, with a prompt like: "Choose a random number", the base pretrained model can give relatively random answers, but after fine tuning to produce responses humans like, they become very biased towards responding with numbers like "7" or "42".

Why is that ? Whenever I’m giving examples I almost always use 7, something ending in a 7 or something in the 70s

My guess is that we bias towards numbers with cultural or personal significance. 7 is lucky in western cultures and is religiously significant (see https://en.wikipedia.org/wiki/7#Culture). 42 is culturally significant in science fiction, though that's a lot more recent. There are probably other examples, but I imagine the mean converges on numbers with multiple cultural touchpoints.

Re: People are just as bad as my LLMs

#13

Post by da_chicken »

aidos wrote: post #8
tehsauce wrote: post #3

There has been some good research published on this topic of how RLHF, ie aligning to human preferences easily introduces mode collapse and bias into models. For example, with a prompt like: "Choose a random number", the base pretrained model can give relatively random answers, but after fine tuning to produce responses humans like, they become very biased towards responding with numbers like "7" or "42".

Why is that ? Whenever I’m giving examples I almost always use 7, something ending in a 7 or something in the 70s

The theory I've heard is that the more prime a number is, the more random it feels. 13 feels more awkward and weird, and it doesn't come up naturally as often as 2 or 3 do in everyday life. It's rare, so it must be more random! I'll give you the most random number I can think of!

People tend to avoid extremes, too. If you ask for a number between 1 and 10, people tend to pick something in the middle. Somehow, the ordinal values of the range seem less likely.

Additionally, people tend to avoid numbers that are in other ranges. Ask for a number from 1 to 100, and it just feels wrong to pick a number between 1 and 10. They asked for a number between 1 and 100. Not this much smaller range. You don't want to give them a number they can't use. There must be a reason they said 100. I wonder if the human RNG would improve if we started asking for numbers between 21 and 114.

Re: People are just as bad as my LLMs

#14

Post by th0ma5 »

mdp2021 wrote: post #9

Very nice article. But the title, and the idea, is the very frequent "racist" form of the proper "People [can be] just as bad as my LLMs". Now: some people can't count. Some people hum between words. Some people set fire to national monuments. Reply: "Yes we knew", and "No, it's not necessary". And: if people could lift the tons, we would not have invented cranes. Very, very often in these pages I meet people repeati…

Racist is the wrong word probably maybe ... antisocial in that it is against society.

Re: People are just as bad as my LLMs

#15

Post by moffkalast »

tehsauce wrote: post #3

There has been some good research published on this topic of how RLHF, ie aligning to human preferences easily introduces mode collapse and bias into models. For example, with a prompt like: "Choose a random number", the base pretrained model can give relatively random answers, but after fine tuning to produce responses humans like, they become very biased towards responding with numbers like "7" or "42".

It's very funny that people hold the autoregressive nature of LLMs against them, while being far more hardline autoregressive themselves. It's just not consciously obvious.

Re: People are just as bad as my LLMs

#16

Post by d0liver »

aidos wrote: post #8
tehsauce wrote: post #3

There has been some good research published on this topic of how RLHF, ie aligning to human preferences easily introduces mode collapse and bias into models. For example, with a prompt like: "Choose a random number", the base pretrained model can give relatively random answers, but after fine tuning to produce responses humans like, they become very biased towards responding with numbers like "7" or "42".

Why is that ? Whenever I’m giving examples I almost always use 7, something ending in a 7 or something in the 70s

I like prime numbers. Non-primes always feel like they're about to fall apart on me.

Re: People are just as bad as my LLMs

#17

Post by bxguff »

Kind of an odd metric to try to base this process off of. are more comments inherently better? is it responding to buzz words? Makes sense talking about hiring algos / resume scanners in part one and if anything this elucidates some of the trouble with them.

Re: People are just as bad as my LLMs

#18

Post by harrisonjackson »

devit wrote: post #4

The "person one" vs "person two" bias seems trivially solvable by running each pair evaluation twice with each possible labelling and the averaging the scores. Although of course that behavior may be a signal that the model is sort of guessing randomly rather than actually producing a signal.

Agreed on the second part. Correcting for bias this way might average out the scores but not in a way that correctly evaluates the HN comments.

The LLM isn't performing the desired task.

It sounds possible to cancel out the comments where reversing the labels swaps the outcome because of bias. That will leave the more "extreme" HN comments that it consistently scored regardless of the label. But that may not solve for the intended task still.

Re: People are just as bad as my LLMs

#19

Post by markbergz »

For anyone interested in these LLM pairwise sorting problems, check out this paper: https://arxiv.org/abs/2306.17563

The authors discuss the person 1 / doc 1 bias and the need to always evaluate each pair of items twice.

If you want to play around with this method there is a nice python tool here: https://github.com/vagos/llm-sort

Re: People are just as bad as my LLMs

#20

Post by smallnix »

Is my understanding wrong that LLMs are trained to emulate observed human behavior in their training data?

From that follows that LLMs fit to produce all kinds of human biases. Like preferring the first choice out of many, and the last our of many (primacy biases). Funnily the LLM might replicate the biases slightly wrong and by doing so produce new derived biases.

Post reply on HN