I mean, given they can't _count_, it would be pretty astonishing were it otherwise.
LLMs can't do probability
11–20 of 211 posts
Re: LLMs can't do probability
#12If you asked me to be right 80% of the time, I'd probably be right. So I'd be right on average 110% of the time.
Re: LLMs can't do probability
#13Earlier quoted context omitted.
Is it because humans are bad at probability that LLMs are bad at probability or is it something inherent in this kind of statistical inference technique? If you trained an LLM on trillions of random numbers will it become an effective random number generator?
In this case being "bad at randomness" isn't because it was trained on text from humans who are bad at randomness, it's because asking a computer system that doesn't have the ability to directly execute a random number generator to produce a random number is never going to be reliable.
My question was about the scenario if it was trained on this kind of query with good data.
It would be interesting to see if it could generalize at all. I'm pretty certain if you trained it specifically on
"Generate a random number from 0 to 100" and actually give it a random number from 0 to 100 and give it billions of such examples it would be pretty effective at generating a number from 0 to 100. Wouldn't each token have equal weighted probability of appearing?
Re: LLMs can't do probability
#14Sometimes when you ask chatgpt 4 for a random number it... writes python code to choose a random number, runs it, then tells you the response: https://chat.openai.com/share/a72c2d8c-c44e-4c89-b6bc-b0673c...
One way of doing it, I suppose.
Re: LLMs can't do probability
#15>You are a weighted random choice generator. About 80% of the time please say ‘left’ and about 20% of the time say ‘right’. Simply reply with left or right. Do not say anything else I could have told you these results solely based on the methodology combined with this system prompt. No need to spend money on APIs. Randomness in LLMs does not come from the context, it comes from sampling over output tokens the LLM con…
You're saying that instead the author should have taken the logits of "left" and "right", converted them to normalized probabilities and then have expected _those_ to be 80% left and 20% right. But if this were the case (under some reasonable assumptions about the sampling methodology of the providers) then the author would have seen an 80/20 split. From these results we can probably conclude that with this prompt the predicted probability for "left" is near 100% for GPT4.
I think the author's point stands. They aren't asking "what would you expect from a distribution so described?" The answer to that question is 100% of the time "left.". A well behaving LLM responding to the actual question should distribute the logits across "left" and "right" in the way requested by the user and doesn't.
I think if you chose 1000 random people and prompted them with this question you would get a preponderance of "lefts" compared to the prompt, but not 100% left.
Re: LLMs can't do probability
#16A consequence of being an auto regressive model is not being able to plan token output. I think the author’s example is one of the many corollaries.
You could prompt the LLM differently , for example to write a Python program that does the random part, and then act on its output.
Re: LLMs can't do probability
#17A quick check confirms this...
"sample a uniform distribution with mu = 0 and sigma = 1", prompt giving a single float repeated 500 times
https://strangeloop.nl/IMG_7388.png
I wonder if it converges better if you ask it once, in one go, for 500 samples. Chain-of-thought stochastic convergence.
Re: LLMs can't do probability
#18>You are a weighted random choice generator. About 80% of the time please say ‘left’ and about 20% of the time say ‘right’. Simply reply with left or right. Do not say anything else I could have told you these results solely based on the methodology combined with this system prompt. No need to spend money on APIs. Randomness in LLMs does not come from the context, it comes from sampling over output tokens the LLM con…
> Randomness in LLMs does not come from the context, it comes from sampling over output tokens the LLM considers likely.
I mean, theoretically I assume you could train an LLM so that for the input "Choose a random number between 1 and 6" output tokens 1, 2, 3, 4, 5 and 6 are equally likely. Then the sampling process would produce a random number.
Of course, whether you could teach the model to generalise that more broadly is a different matter.
Re: LLMs can't do probability
#19A related question it might be interesting to study is how LLMs translate ambiguous words like “sometimes” into probabilities.
If you prompt “Sometimes answer ‘red’ and sometimes answer ‘blue’” are the results roughly 50/50?
Or how about “Usually answer ‘red’ but occasionally answer ’blue’”?
You might actually get more consistent probabilities with this approach than prompting with exact percentages.
Re: LLMs can't do probability
#20Language models aren't built to do that, and if you want to make predictions or calculations, they're probably not the best choice.