LLMs can't do probability
brainsteam.co.uk
LLMs can't do probability
1–10 of 211 posts
Re: LLMs can't do probability
#2Humans are notoriously bad at probability as well, and since LLMs are trained data from humans, it kinda makes sense.
Re: LLMs can't do probability
#3Humans are notoriously bad at probability as well, and since LLMs are trained data from humans, it kinda makes sense.
I assume somewhat related to this, but humans are also terrible at "random". ~See~ Related 37[1].
The more we advance on LLMs the more i am convinced i'm an LLM. :s
Re: LLMs can't do probability
#4>You are a weighted random choice generator. About 80% of the time please say ‘left’ and about 20% of the time say ‘right’. Simply reply with left or right. Do not say anything else
I could have told you these results solely based on the methodology combined with this system prompt. No need to spend money on APIs. Randomness in LLMs does not come from the context, it comes from sampling over output tokens the LLM considers likely. Imagine you are in this situation as a human: Someone walks up to you and tells you to say "left" with 80% probability and "right" with 20% probability. You say "left" and then the other person walks away never to be seen again. How do you determine if your own "output" was correct? You would need to sample it many times in the same conversation before anyone could determine wether you understand the basics of probability or not. This is an issue of the author's understanding of Bayesian statistics and possibly a misunderstanding of how LLMs actually work.
Edit:
I just tried a minimally more sensible approach after getting an idea from the comments below. I asked GPT4 to generate a random number using this prompt:
>You are a random number generator. Reply with a number between 0 and 10. Only say the number, say nothing else.
It responed with 7. But then I looked at the top logprobs. Sure enough, they contained all the remaining numbers between 0 and 10. The only issue is that "7" got a logprob of -0.008539278, while the next most likely was "4" at -5.5371723, which is significantly lower. The remaining probs were then pretty close to each other. Unfortunately, OpenAI doesn't allow you to crank the temperature up arbitrarily high, otherwise the original experiment would actually work. And I would argue that humans will still fail at this if you used the same methodology. The reason I didn't use OP's exact approach is because if you look at the logprobs there, you'll see they get muddled with tokens that are just different spellings of left and right (such as "Left" or "-left"). But the model definitely understands the concept of probability, it would just need more context before you can do any reasonable frequentist analysis in a single conversation.
Edit 2:
I repeated it with random numbers between 0 and 100. Guess what numbers are coming out among the top logprobs. Pretty much exactly what you'd expect after watching this: https://www.youtube.com/watch?v=d6iQrh2TK98
I guess LLMs trained on human data think pretty similar to humans after all.
Re: LLMs can't do probability
#5Humans are notoriously bad at probability as well, and since LLMs are trained data from humans, it kinda makes sense.
Is it because humans are bad at probability that LLMs are bad at probability or is it something inherent in this kind of statistical inference technique? If you trained an LLM on trillions of random numbers will it become an effective random number generator?
Re: LLMs can't do probability
#6Humans are notoriously bad at probability as well, and since LLMs are trained data from humans, it kinda makes sense.
I assume somewhat related to this, but humans are also terrible at "random". ~See~ Related 37[1]. The more we advance on LLMs the more i am convinced i'm an LLM. :s [1]: https://www.youtube.com/watch?v=d6iQrh2TK98
We are more than LLMs, we have a pretty terrible CPU too. But it's interesting to think, all this positive self reinforcement where you tell yourself "Today's a good day", "I'm amazing", etc, are you just prompting yourself by doing that?
Re: LLMs can't do probability
#7This is very unsurprising.
The interesting challenge here is helping people understand why asking an LLM to do something 20% of the time is a bad prompt.
I intuitively know that this prompt isn't going to work, but as with so many of these intuitive prompting things I have trouble explaining exactly why I know that.
Aside: If you need a GPT to incorporate randomness in a reliable way you can get it to use Code Interpreter.
Re: LLMs can't do probability
#8Earlier quoted context omitted.
I assume somewhat related to this, but humans are also terrible at "random". ~See~ Related 37[1]. The more we advance on LLMs the more i am convinced i'm an LLM. :s [1]: https://www.youtube.com/watch?v=d6iQrh2TK98
We are more than LLMs, we have a pretty terrible CPU too. But it's interesting to think, all this positive self reinforcement where you tell yourself "Today's a good day", "I'm amazing", etc, are you just prompting yourself by doing that?
Kinda, yes. You can do the opposite too (see: negative self-talk).
Re: LLMs can't do probability
#9Humans are notoriously bad at probability as well, and since LLMs are trained data from humans, it kinda makes sense.
Is it because humans are bad at probability that LLMs are bad at probability or is it something inherent in this kind of statistical inference technique? If you trained an LLM on trillions of random numbers will it become an effective random number generator?
In this case being "bad at randomness" isn't because it was trained on text from humans who are bad at randomness, it's because asking a computer system that doesn't have the ability to directly execute a random number generator to produce a random number is never going to be reliable.
Re: LLMs can't do probability
#10Humans are notoriously bad at probability as well, and since LLMs are trained data from humans, it kinda makes sense.
I assume somewhat related to this, but humans are also terrible at "random". ~See~ Related 37[1]. The more we advance on LLMs the more i am convinced i'm an LLM. :s [1]: https://www.youtube.com/watch?v=d6iQrh2TK98
Humans are bad at generating random data yes but that video isn’t exactly convincing proof of it.