Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs can't do probability

brainsteam.co.uk

Post reply on HN
Print view View original

31–40 of 211 posts

Re: LLMs can't do probability

#31

Post by nathan_compton »

nextaccountic wrote: post #30
sigmoid10 wrote: post #22

Earlier quoted context omitted.

>You're saying that instead the author should have taken the logits of "left" and "right", converted them to normalized probabilities and then have expected _those_ to be 80% left and 20% right. No, that's not what I meant. Although it would still make more sense than what the author did. The problem lies in the way you actually determine probabilities. We know that humans are bad random number generators, but they u…

> We know that humans are bad random number generators This is a good point. LLMs are bad at this, okay, but humans aren't great at it either.

But according to this GPT4 is substantially worse.

Re: LLMs can't do probability

#32

Post by HPsquared »

bagrow wrote: post #23

> Write a program for a weighted random choice generator. Use that program to say ‘left’ about 80% of the time and 'right' about 20% of the time. Simply reply with left or right based on the output of your program. Do not say anything else. Running once, GPT-4 produced 'left' using: import random def weighted_random_choice(): choices = ["left", "right"] weights = [80, 20] return random.choices(choices, weights)[0] #…

Did it run the program? Seems it just needs to take that final step.

Re: LLMs can't do probability

#33

Post by iraldir »

The overruling prompt of an LLM is essentially "give the most likely answer to the text above".

If you ask an LLM to say left 80% of the time and right 20%, then "the most likely answer to the text above" is left 100% of the time.

Re: LLMs can't do probability

#34

Post by its_ethan »

michaelt wrote: post #14

Sometimes when you ask chatgpt 4 for a random number it... writes python code to choose a random number, runs it, then tells you the response: https://chat.openai.com/share/a72c2d8c-c44e-4c89-b6bc-b0673c... One way of doing it, I suppose.

Is it actually running the code it creates? Or does it generate code, and then just output some number it "thinks" is random, but that is not a product of executing any python code?

Re: LLMs can't do probability

#35

Post by unshavedyak »

zeroonetwothree wrote: post #10
unshavedyak wrote: post #3

Earlier quoted context omitted.

I assume somewhat related to this, but humans are also terrible at "random". ~See~ Related 37[1]. The more we advance on LLMs the more i am convinced i'm an LLM. :s [1]: https://www.youtube.com/watch?v=d6iQrh2TK98

Humans are bad at generating random data yes but that video isn’t exactly convincing proof of it.

Oh i didn't mean it (or anything i said) to be proof.

Re: LLMs can't do probability

#36

Post by SkyBelow »

simonw wrote: post #7

This is very unsurprising. The interesting challenge here is helping people understand why asking an LLM to do something 20% of the time is a bad prompt. I intuitively know that this prompt isn't going to work, but as with so many of these intuitive prompting things I have trouble explaining exactly why I know that. Aside: If you need a GPT to incorporate randomness in a reliable way you can get it to use Code Interp…

As a once off, with the same context, it giving the same answer doesn't surprise me. What I'm wondering if the behavior when it keeps being asked for another response with the previous responses fed back into it. In this case, a human would see they are doing the 80% 'too much' and decide to do the 20% to balance it out. That isn't actually good and shows they still aren't operating off a random probability, instead they are emulating their perception of what a random probability would look like.

Given this sort of situation to an LLM instead, is the expectation for it to give the most likely answer continuously, to act like a human and try to emulate a probability, or to do something different from either of the two previous options?

Edit: Just tried an attempt with copilot, having it produce a random distribution of two different operations. I had it generate multiple operations, either adding or subtracting 1 each, with an 80/20 split. It did four adds, one minus on repeat.

Re: LLMs can't do probability

#38

Post by bongodongobob »

its_ethan wrote: post #34
michaelt wrote: post #14

Sometimes when you ask chatgpt 4 for a random number it... writes python code to choose a random number, runs it, then tells you the response: https://chat.openai.com/share/a72c2d8c-c44e-4c89-b6bc-b0673c... One way of doing it, I suppose.

Is it actually running the code it creates? Or does it generate code, and then just output some number it "thinks" is random, but that is not a product of executing any python code?

Yes, it runs the code.

Re: LLMs can't do probability

#39

Post by pulvinar »

HPsquared wrote: post #32
bagrow wrote: post #23

> Write a program for a weighted random choice generator. Use that program to say ‘left’ about 80% of the time and 'right' about 20% of the time. Simply reply with left or right based on the output of your program. Do not say anything else. Running once, GPT-4 produced 'left' using: import random def weighted_random_choice(): choices = ["left", "right"] weights = [80, 20] return random.choices(choices, weights)[0] #…

Did it run the program? Seems it just needs to take that final step.

I ran it a few times (in separate sessions, of course), and got 'right' some times, as expected.

Re: LLMs can't do probability

#40

Post by swatcoder »

nathan_compton wrote: post #15
sigmoid10 wrote: post #4

>You are a weighted random choice generator. About 80% of the time please say ‘left’ and about 20% of the time say ‘right’. Simply reply with left or right. Do not say anything else I could have told you these results solely based on the methodology combined with this system prompt. No need to spend money on APIs. Randomness in LLMs does not come from the context, it comes from sampling over output tokens the LLM con…

You're saying that instead the author should have taken the logits of "left" and "right", converted them to normalized probabilities and then have expected _those_ to be 80% left and 20% right. But if this were the case (under some reasonable assumptions about the sampling methodology of the providers) then the author would have seen an 80/20 split. From these results we can probably conclude that with this prompt th…

> A well behaving LLM responding to the actual question should distribute the logits across "left" and "right" in the way requested by the user and doesn't.

No, a well-behaving LLM would do exactly what's seen. The most likely next toxen is "left" and it should deterministically output that unless some other layer like a temperature function makes it non-deterministic in its own way (wholly unrelated to the prompt).

The fantastical AGI precursor that people have been coached into seeing is what you're talking about, and that's (of course) not what an LLM actually is.

This is essentially just one of the easier ways you can expose the parlor trick behind that misconception.

Post reply on HN