Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Creativity has left the chat: The price of debiasing language models

arxiv.org

Post reply on HN
Print view View original

171–180 of 238 posts

Re: Creativity has left the chat: The price of debiasing language models

#171

Post by cratermoon »

sgt101 wrote: post #69
anewhnaccount3 wrote: post #44

There is a bit of a false equivalence between entropy of output distributions and creativity here. Is diversity really the same as creativity?

No, diversity isn't creativity. For example, we could search google for "great art" and if it produced a sample of one art work from ever decade of the last 500 years that would likely be highly diverse in style and content. If it returned a list of the best work from western Europe in the of the 18th century it would be rather consistent. Both lists would have the same amount of creativity though - 0.

"one art work from every decade of the last 500 years that would likely be highly diverse in style and content"

It still might not be especially diverse if all 50 examples were from western European art. 500 years only takes us back to 1524 - not especially long and mostly from the same early modern period starting with the fall of Constantinople, the end of the Crusades, and the start of the Renaissance. I wouldn't be surprised if 80% or more of the works ended up being some depiction of aspects of Christianity painted by a white male.

Re: Creativity has left the chat: The price of debiasing language models

#172

Post by robertlagrant »

cratermoon wrote: post #171
sgt101 wrote: post #69

Earlier quoted context omitted.

No, diversity isn't creativity. For example, we could search google for "great art" and if it produced a sample of one art work from ever decade of the last 500 years that would likely be highly diverse in style and content. If it returned a list of the best work from western Europe in the of the 18th century it would be rather consistent. Both lists would have the same amount of creativity though - 0.

"one art work from every decade of the last 500 years that would likely be highly diverse in style and content" It still might not be especially diverse if all 50 examples were from western European art. 500 years only takes us back to 1524 - not especially long and mostly from the same early modern period starting with the fall of Constantinople, the end of the Crusades, and the start of the Renaissance. I wouldn't…

> I wouldn't be surprised if 80% or more of the works ended up being some depiction of aspects of Christianity painted by a white male.

Are you saying diversity in art is signified by the artist's race and sex?

Re: Creativity has left the chat: The price of debiasing language models

#173

Post by throwup238 »

kmeisthax wrote: post #17
MrThoughtful wrote: post #8

How hard would it be to create a "raw" model on a corpus like Hacker News or Wikipedia? With "raw", I mean that it is simply trained to predict the next token and nothing else. Would be fun to play with such a model.

You want a pure-human training data set, so you have to go back in time to before 2020 to scrape training data. Either that, or only use data with a verified Wayback machine capture from before 2020. Or invent a new training regime that doesn't require gobs of stolen text. Actually, I have a bit of a hunch that the publishers currently suing IA over their unlicensed digital library lending program plan to bankrupt it…

You can just use Common Crawl. They have archives of their scrape data going back to 2008.

Re: Creativity has left the chat: The price of debiasing language models

#174

Post by throwaway22032 »

Okay, so as a thought experiment, let's say we get a superintelligent LLM, capable of somehow connecting the dots and knowing more than us as humans.

How do we avoid interpreting its correct results as bias? I mean, what do we do when it tells us that (fake example) IQ is correlated with height and that people above 6ft are more intelligent?

I'm sure you can think of spicier examples. Will we try to "debias" it by encouraging it to spit out incorrect information or just ignore certain topics?

Re: Creativity has left the chat: The price of debiasing language models

#175

Post by pulvinar »

cratermoon wrote: post #169
lispisok wrote: post #10

Is this why all the coding AI products I've used have gotten worse as the developers fine tune them to eliminate bad output? Before there was bad output and some interesting output, now it's just bland obvious stuff.

It's not always possible to say definitely is some text was AI-generated or not, but one sign that it is very likely AI is a kind of blandness of affect. Even marketing text carefully written by humans to avoid offensiveness tends to exude a kind of breathless enthusiasm for whatever it's selling. If marketing text is oatmeal with raisins, AI text is plain oatmeal. It's possible to adjust the output of an LLM with te…

You can ask the LLM "now describe it with breathless enthusiasm", if that's what you want. There's been no shortage of training examples out there.

Re: Creativity has left the chat: The price of debiasing language models

#176

Post by wongarsu »

jrm4 wrote: post #160

"Bias" implies the possibility of "unbiased language model" which seems to be in the category of things that are on one hand, COMPLETELY IMPOSSIBLE, and on the other, still likely to be sold on the market because market wants it so much?

Even assuming we can make an unbiased model (assuming by unbiased we mean something like "has a world model and reasoning that has no systematic deviation from reality"), we couldn't recognize the model as unbiased. I'd even wager that outside of research such a model would be completely unusable for practical applications.

Both as individual humans and as collective societies we have a lot of biases. And judging by how fundamental values of societies shift across time and civilizations it's basically guaranteed that an unbiased view (whatever that is) would be incompatible with our views on many basic topics.

What most people want is a language model that matches our biases. Of course we can't even agree on what those are, and which biases are useful (is a bias against telling people how to cook meth or build a bomb good? What about using expletive language?).

Though in this paper I gather "unbiased" just refers to "only the bias acquired by training method and training data, without meddling or fine tuning"

Re: Creativity has left the chat: The price of debiasing language models

#177

Post by sleepybrett »

flohofwoe wrote: post #53
rhcom2 wrote: post #22

Earlier quoted context omitted.

> Authoritarian societies don't produce great creative work. Is that even true though? Off the top of my head I can think of the art of Soviet propaganda posters, Leni Riefenstahl, Liu Cixin.

Eastern European science fiction would be a better example. Authors like Stanislaw Lem or the Strugatski brothers had to adapt to sneak critical ideas past censors, and readers had to adapt and read between the lines. (also, categorizing propaganda posters as art, ewwh...)

on your parenthetical, you can see the artistry in the pure visual expression no matter how loathsome the subject matter.

Re: Creativity has left the chat: The price of debiasing language models

#178

Post by Imnimo »

>T ∈ (0, 1] is a parameter called temperature which controls the “softness” of the probability distribution. In our experiments we choose T = 1.0 for maximum response variation.

Why is temperature bounded to be <=1? If you want more "creativity" out of the chat model, can you just set T higher and recover a similar distribution to the base model?

Re: Creativity has left the chat: The price of debiasing language models

#179

Post by wongarsu »

freehorse wrote: post #157
SirMaster wrote: post #153

Earlier quoted context omitted.

So is the reason why LLMs don't say when they don't know something and instead make up something that "sounds right" because the RLHF has taught it to always give an answer? And if that's the case, why? Is that really what people want an LLM to do? I feel like I would rather it say when it doesn't know something.

LLMs do not know what "they know" or they don't. They just autocomplete what sounds best relevant based on their training set. They do not have enough "I don't know" in their training set in the first place most probably.To have them say "I don't know" you have to go into finetuning them heavily. So, if anything, they hallucinate a lot more without RLHF. Which in this paper they call "creativity".

In the GPT3 days when everyone was doing few-shot tasks (giving the LLM a couple of examples of question/answer pairs in the prompt) one of the big insights was that adding question/answer pairs with answers like "I don't know" and "this question doesn't make sense" caused the model to actually use those answers appropriately instead of overconfidently stating nonsense.

Of course that method isn't perfect (GPT3.0 was far from perfect in general). But both in principle and in practice the models do have a notion of what they "know". Knowledge is a strong activation, random noise is a weaker activation, you "just" have to get the model to override those weaker activations with admitting failure.

You could draw parallels to allowing LLMs to emit pause tokens to get more time to think (https://arxiv.org/abs/2310.02226 and similar). At some level of abstraction that's also just training the model to replace uncertain answers with a special token, in the hope that it eventually reaches more certainty.

Re: Creativity has left the chat: The price of debiasing language models

#180

Post by Der_Einzige »

Imnimo wrote: post #178

>T ∈ (0, 1] is a parameter called temperature which controls the “softness” of the probability distribution. In our experiments we choose T = 1.0 for maximum response variation. Why is temperature bounded to be <=1? If you want more "creativity" out of the chat model, can you just set T higher and recover a similar distribution to the base model?

They'll tell you "No" and say that you ruin your samplers, but good samplers (dynamic ones) like min_p or typicality are robust to high temperatures, so in actuality yes.
Post reply on HN