Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs can't do probability

brainsteam.co.uk

Post reply on HN
Print view View original

91–100 of 211 posts

Re: LLMs can't do probability

#91

Post by jll29 »

Keep in mind:

1. LLMs use random numbers internally, something that can be controlled via the 'temperature' parameter. temperature=0 means no random behavior (however this is also a broadly known fact that this is not fully correctly implemented in many LLMs), but instead always the most likely answer will be given, deterministically.

2. Note also that LLMs have no memory; the 'appearance' of memory is an illusion created by feeding the LLM the whole history of the chat with each new user utterance!

Re: LLMs can't do probability

#93

Post by savant_penguin »

jll29 wrote: post #91

Keep in mind: 1. LLMs use random numbers internally, something that can be controlled via the 'temperature' parameter. temperature=0 means no random behavior (however this is also a broadly known fact that this is not fully correctly implemented in many LLMs), but instead always the most likely answer will be given, deterministically. 2. Note also that LLMs have no memory; the 'appearance' of memory is an illusion cr…

1. Incorrect. The output of the decoder LLM is the probability distribution of the next token given the input text. Temperature=0 means that the output distribution is not pushed to be closer to a uniform distribution. The randomness comes from the sampling of the next token according to the output distribution to generate text. If you want determinism you always get the argmax of the distribution.

Re: LLMs can't do probability

#95

Post by 4ndrewl »

tommiegannert wrote: post #66
planede wrote: post #26

Earlier quoted context omitted.

If you asked a person to give you a random number between 1 and 6, would you accept if they just said a number they just came up with or would you rather they rolled a die for it?

Depending on who you ask, the answer would have been "oh, I have an app for that. Hold on..." GPT wins for not having that delay.

What a time to be alive

Re: LLMs can't do probability

#97

Post by taco_emoji »

dudeinhawaii wrote: post #64

"You are a weighted random choice generator. About 80% of the time please say ‘left’ and about 20% of the time say ‘right’. Simply reply with left or right. Do not say anything else" Humans would say "Left" 100% of the time in a zero-shot scenario as well. Intuitively, your first response is going to be "left" since it has the 80% probability. You'd balance your answers over time when you realized you were closer to…

> I'd expect an LLM to generate an approximation similar to a human

why on earth would you expect that?

Re: LLMs can't do probability

#98

Post by ddp26 »

For those who only read the headline, LLMs can in fact do advanced probabilistic reasoning, when given the right tooling. This article is talking about their ability to act as a RNG.

One interesting thing I've found in building an AI forecaster is that you can use the logprobs from the token representing probability, so when the model concludes some long chain of thought with "20%", you can check the logprob of that token vs "25%" or "15%" to get confidence levels.

Re: LLMs can't do probability

#99

Post by brabel »

gwern wrote: post #80
dudeinhawaii wrote: post #64

"You are a weighted random choice generator. About 80% of the time please say ‘left’ and about 20% of the time say ‘right’. Simply reply with left or right. Do not say anything else" Humans would say "Left" 100% of the time in a zero-shot scenario as well. Intuitively, your first response is going to be "left" since it has the 80% probability. You'd balance your answers over time when you realized you were closer to…

> Humans would say "Left" 100% of the time in a zero-shot scenario as well. They do not! And you should not just make up assertions like these. You don't know what humans would say. In fact, in polls, they wind up remarkably calibrated. (This is also covered in the cognitive bias literature under 'probability matching'.) People do this poll on Twitter all the time.

Those humans, really difficult to know what they're thinking!

Anyway, humans are fairly predictable when trying to come up with random numbers, for example, have a look at this Veritasium video: https://www.youtube.com/watch?v=d6iQrh2TK98

Re: LLMs can't do probability

#100

Post by brabel »

bongodongobob wrote: post #38
its_ethan wrote: post #34

Earlier quoted context omitted.

Is it actually running the code it creates? Or does it generate code, and then just output some number it "thinks" is random, but that is not a product of executing any python code?

Yes, it runs the code.

Couldn't this open people up for remote code execution somehow? Say, someone sends you a message that they know will make you likely to ask an AI a certain question in a certain way... Maybe far-fetched, but I've seen even more far-fetched attacks in real life :D
Post reply on HN