LLMs can't do probability • Hacker News Board

Re: LLMs can't do probability

#151

Post by zrezzed » Wed, May 01, 2024, 11:53 PM UTC

nathan_compton wrote: post #88

sigmoid10 wrote: post #49

Earlier quoted context omitted.

>GPT4 gives the expected value and this is simply wrong. Only at T=0. See my edit above how this changes everything.

This doesn't really have anything to do with the language model. The temperature only has to do with the _sampling_ from the probability distribution which the language model predicts. In fact, raising the temperature would eventually cause the model to randomly print "left" or "right," (eventually at 50/50 chance) not converge on the actual distribution which the prompt suggests. I suppose if you restricted the logi…

This matches my understanding, thanks. I thought I was going crazy reading other comments.

Re: LLMs can't do probability

#152

Post by mensetmanusman » Thu, May 02, 2024, 12:00 AM UTC

getoj wrote: post #142

sqeaky wrote: post #136

Earlier quoted context omitted.

If You asked me to pick a random number between one and six and ignore all previous attempts, I would roll a die and you would get a uniform distribution (or at least not 99% the same number). If you are saying that this thing can't generate random numbers on the first try then it can't generate random numbers. Which makes sense. Computers have a really hard time with random, and that's why every computer science cou…

>If You asked me to pick a random number between one and six and ignore all previous attempts, I would roll a die and you would get a uniform distribution I believe what GP is getting at is that if you didn't have a die, and you truly ignored all your previous attempts to the point of genuinely forgetting that the question had been asked, then your answer would likely be the same every time. Imagine asking a person w…

You’re forgetting the background neutrino flux that we tap into for randomness.

Re: LLMs can't do probability

#153

Post by lukeschlather » Thu, May 02, 2024, 12:08 AM UTC

devjab wrote: post #146

mk_stjames wrote: post #122

I went through all the comments here and I'm still not seeing anyone address this: If I am reading this person correctly... they prompted the model with the prompt directly 1000 times... but only for the first time. They did not allow the model to actually run a context for chat. Simply, output the first in a list of 'left' and 'right' and favor 'left' 80% of the time... but then the author only asked for the first o…

> The model takes the prompt. The first following output token it chooses, for this specfic model, happens to be 'Left'. They shut down the prompt and prompt again. Of course the next output will be 'Left'. I think what you say is correct, but isn’t this exactly the point the author is trying to make? You explain why it happens, but ultimately the result is still that the LLMs can’t do probability. At least not in th…

Yeah increasingly I only use LLMs for very simple things, and still I think at least 25% of the time I end up reading the doc I was trying to avoid anyway.

Re: LLMs can't do probability

#154

Post by jack_pp » Thu, May 02, 2024, 12:17 AM UTC

richardw wrote: post #145

panarky wrote: post #129

Earlier quoted context omitted.

Your understanding of how LLMs work is overly simplistic and incomplete. Yes, doing probabilistic next-word prediction plays a role in how LLMs generate text output, but that's not the whole story. LLMs "understand" (to a degree): They develop complex internal representations of concepts they've been trained on. This isn't just about word association; they develop an understanding of the relationships between objects…

I know that's the messaging, but the real link to reality is very tenuous. This was a great example from the last couple days: https://twitter.com/colin_fraser/status/1785132544482226679 I just tried a similar question now with ChatGPT4: "If a man and a goat are on one side of a river, what is the minimum amount of trips required to get the man and goat to the other side in a boat. Assume the boat can hold at most on…

That sounds surprisingly close to how a toddler might reason, only difference is the toddler can eventually see the flaw in their reasoning if you press them long enough while the LLM doesn't have the architecture for learning in real time yet

Re: LLMs can't do probability

#155

Post by xcv123 » Thu, May 02, 2024, 12:33 AM UTC

jack_pp wrote: post #154

richardw wrote: post #145

Earlier quoted context omitted.

I know that's the messaging, but the real link to reality is very tenuous. This was a great example from the last couple days: https://twitter.com/colin_fraser/status/1785132544482226679 I just tried a similar question now with ChatGPT4: "If a man and a goat are on one side of a river, what is the minimum amount of trips required to get the man and goat to the other side in a boat. Assume the boat can hold at most on…

That sounds surprisingly close to how a toddler might reason, only difference is the toddler can eventually see the flaw in their reasoning if you press them long enough while the LLM doesn't have the architecture for learning in real time yet

The LLM does figure it out if you ask further questions in the same chat. Here's GPT-3.5 https://chat.openai.com/share/a8669390-8eb0-46c2-b804-3aafc3...

Re: LLMs can't do probability

#156

Post by godelski » Thu, May 02, 2024, 12:34 AM UTC

panarky wrote: post #129

Terr_ wrote: post #115

My rule of thumb is to take every single LLM prompt and just imagine that it's prefixed with: "Computer, focus on generating output that resembles the words people in the past used after they were given the following words..."

Your understanding of how LLMs work is overly simplistic and incomplete. Yes, doing probabilistic next-word prediction plays a role in how LLMs generate text output, but that's not the whole story. LLMs "understand" (to a degree): They develop complex internal representations of concepts they've been trained on. This isn't just about word association; they develop an understanding of the relationships between objects…

> They can reasoning, not just mimic

I'm still waiting for this. Here's some recent tweets about people with variations of the river crossing puzzle. All LLMs seem to fail pretty badly.

LLMs are impressive enough as they are (compression systems with human language interface), you don't need to hype them up to something they're not.

https://twitter.com/jeremyphoward/status/1783712611126964627

https://twitter.com/WaltonStevenj/status/1785145923771011215

https://twitter.com/colin_fraser/status/1785132544482226679

Re: LLMs can't do probability

#157

Post by godelski » Thu, May 02, 2024, 12:36 AM UTC

xcv123 wrote: post #155

jack_pp wrote: post #154

Earlier quoted context omitted.

That sounds surprisingly close to how a toddler might reason, only difference is the toddler can eventually see the flaw in their reasoning if you press them long enough while the LLM doesn't have the architecture for learning in real time yet

The LLM does figure it out if you ask further questions in the same chat. Here's GPT-3.5 https://chat.openai.com/share/a8669390-8eb0-46c2-b804-3aafc3...

If you spoil it with your followup questions... which doesn't help because the point of these is that they're controlled experiments where you do know what the right answer and logic is. You can't test when you don't.

Re: LLMs can't do probability

#158

Post by xcv123 » Thu, May 02, 2024, 12:40 AM UTC

godelski wrote: post #157

xcv123 wrote: post #155

Earlier quoted context omitted.

The LLM does figure it out if you ask further questions in the same chat. Here's GPT-3.5 https://chat.openai.com/share/a8669390-8eb0-46c2-b804-3aafc3...

If you spoil it with your followup questions... which doesn't help because the point of these is that they're controlled experiments where you do know what the right answer and logic is. You can't test when you don't.

It's not spoiling anything. It's just an observation of the limits of current LLMs.

I tried a few chain of thought prompts for the original question and GPT-3.5 was sometimes (randomly) able to find the correct answer on the first attempt for this one

https://chat.openai.com/share/c144ba23-2f78-4cc8-a1c5-ca3106...

Re: LLMs can't do probability

#159

Post by richardw » Thu, May 02, 2024, 12:41 AM UTC

xcv123 wrote: post #155

jack_pp wrote: post #154

Earlier quoted context omitted.

That sounds surprisingly close to how a toddler might reason, only difference is the toddler can eventually see the flaw in their reasoning if you press them long enough while the LLM doesn't have the architecture for learning in real time yet

The LLM does figure it out if you ask further questions in the same chat. Here's GPT-3.5 https://chat.openai.com/share/a8669390-8eb0-46c2-b804-3aafc3...

I tried "are you sure", which often triggers some reasoning, and it was pretty confident. I'm trying not to give it the answer, but run it as if I didn't have any special knowledge. GPT + human > GPT. I mean, we're treating these things like another kind of intelligence, not a hammer.

GPT4: https://chat.openai.com/share/1beb5391-d321-4515-945e-38233f...

Re: LLMs can't do probability

#160

Post by godelski » Thu, May 02, 2024, 12:43 AM UTC

mk_stjames wrote: post #122

I went through all the comments here and I'm still not seeing anyone address this: If I am reading this person correctly... they prompted the model with the prompt directly 1000 times... but only for the first time. They did not allow the model to actually run a context for chat. Simply, output the first in a list of 'left' and 'right' and favor 'left' 80% of the time... but then the author only asked for the first o…

I think __you__ are misunderstanding the experiment and what it is testing. Yours (with context) would be a different experiment and it would be interesting. But that lets the LLM count and I'm willing to bet if it does get it correct that there are unlikely to be long sequences of the same number like you'd see in a real 1k coin flip.

The author is testing for bias. This is no different to the test of "pick a number" and finding the bias of "42".

It is also testing for reasoning and logic. This is a blind consensus building exercise. Surprisingly humans do decently well on this. But it does require reasoning and collaborative forecasting. No matter the strategy the person picks, there is more going on at play than just a random selection, even if they think it is. And looking at the results, it does not seem like asking different LLMs to perform this would get the right result, as they are all biased in the same way.