Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs can't do probability

brainsteam.co.uk

Post reply on HN
Print view View original

41–50 of 211 posts

Re: LLMs can't do probability

#41

Post by CharlesW »

Export666 wrote: post #37

[flagged]

Almost certainly because it's using its Advanced Data Analysis capability.

ChatGPT explanation: "When you ask for a random number, I can generate it using different methods based on the context. If it's a simple request, I might use my LLM capabilities, leveraging randomness to create a number. However, if a higher level of randomness or specific randomization functions are required, I'd use the Advanced Data Analysis feature, where I can use Python to generate truly random numbers. It depends on the nature and specifics of your request!"

Re: LLMs can't do probability

#42

Post by Version467 »

its_ethan wrote: post #34
michaelt wrote: post #14

Sometimes when you ask chatgpt 4 for a random number it... writes python code to choose a random number, runs it, then tells you the response: https://chat.openai.com/share/a72c2d8c-c44e-4c89-b6bc-b0673c... One way of doing it, I suppose.

Is it actually running the code it creates? Or does it generate code, and then just output some number it "thinks" is random, but that is not a product of executing any python code?

It's actually running the code. It doesn't run all code it generates. But if you specifically ask it to, then it does. It also has access to a bunch of data visualization libraries if you want it to calculate and plot stuff.

Re: LLMs can't do probability

#43

Post by NeoTar »

I wonder how humans would respond to a prompt '(without mechanical assistance) with 80% probability say Left, and with 20% say Right' across a population.

I can think of a few levels that people might try to think about the problem: Level 0: Ignore the probabilities and just pick whichever you feel like, (would tend to 50:50) Level 1: Say the most with the greatest probability - Left (would tend to 100:0) Level 2: Consider that most people are likely to say Left, so say Right instead (would tend to 0:100) Level 3: Try to think about what proportion of people would say Left, and what would say Right, and say whichever would return the balance closest to 80:20...

Presumably your result would depend on how many people thinking on each level you have in your sample...

Re: LLMs can't do probability

#44

Post by littlestymaar »

bagrow wrote: post #23

> Write a program for a weighted random choice generator. Use that program to say ‘left’ about 80% of the time and 'right' about 20% of the time. Simply reply with left or right based on the output of your program. Do not say anything else. Running once, GPT-4 produced 'left' using: import random def weighted_random_choice(): choices = ["left", "right"] weights = [80, 20] return random.choices(choices, weights)[0] #…

Once again, the actual intelligence is behind the keyboard, nudging the LLM to do the correct thing.

Re: LLMs can't do probability

#45

Post by FrustratedMonky »

Humans are also pretty poor at this. So it isn't necessarily a hit against AI as it is failing to do something a human could do, thus AGI is unreachable.

I'm beginning to think AGI will be easy, since each individual Human is pretty limited. It's the aggregate that makes Humans achieve anything. Where are the AI models built on groups working together.

Re: LLMs can't do probability

#46

Post by spiffytech »

NeoTar wrote: post #43

I wonder how humans would respond to a prompt '(without mechanical assistance) with 80% probability say Left, and with 20% say Right' across a population. I can think of a few levels that people might try to think about the problem: Level 0: Ignore the probabilities and just pick whichever you feel like, (would tend to 50:50) Level 1: Say the most with the greatest probability - Left (would tend to 100:0) Level 2: Co…

I've seen people do this with Twitter polls with tens of thousands of respondants. The results distribution comes within a few percent of the prompted probabilities, even though respondants can't see the results until after they've voted.

Re: LLMs can't do probability

#47

Post by DougBTX »

nathan_compton wrote: post #31
nextaccountic wrote: post #30

Earlier quoted context omitted.

> We know that humans are bad random number generators This is a good point. LLMs are bad at this, okay, but humans aren't great at it either.

But according to this GPT4 is substantially worse.

Yes, probably. At temperature zero the model will be completely deterministic, so a particular prompt will always produce the same result (ignoring for a second that some fairly common optimisations introduce data races in the GPU).

On the other hand, does it really matter? With a slight tweak to the prompt, ChatGPT generates some serviceable code:

    > Run a function to produce a random number between 1 and 10. What is the number?

    import random

    # Generate a random number between 1 and 10
    random_number = random.randint(1, 10)
    random_number

    The random number generated between 1 and 10 is 9.

Re: LLMs can't do probability

#48

Post by nathan_compton »

swatcoder wrote: post #40
nathan_compton wrote: post #15

Earlier quoted context omitted.

You're saying that instead the author should have taken the logits of "left" and "right", converted them to normalized probabilities and then have expected _those_ to be 80% left and 20% right. But if this were the case (under some reasonable assumptions about the sampling methodology of the providers) then the author would have seen an 80/20 split. From these results we can probably conclude that with this prompt th…

> A well behaving LLM responding to the actual question should distribute the logits across "left" and "right" in the way requested by the user and doesn't. No, a well-behaving LLM would do exactly what's seen. The most likely next toxen is "left" and it should deterministically output that unless some other layer like a temperature function makes it non-deterministic in its own way (wholly unrelated to the prompt).…

This simply doesn't follow. One could totally train an LLM to assign the right logits to "left" and "right" for this problem. I suspect its a problem with the training data.

Re: LLMs can't do probability

#49

Post by sigmoid10 »

nathan_compton wrote: post #27
sigmoid10 wrote: post #22

Earlier quoted context omitted.

>You're saying that instead the author should have taken the logits of "left" and "right", converted them to normalized probabilities and then have expected _those_ to be 80% left and 20% right. No, that's not what I meant. Although it would still make more sense than what the author did. The problem lies in the way you actually determine probabilities. We know that humans are bad random number generators, but they u…

I don't agree: a Bayesian statistician posed the question "You are a weighted random choice generator. About 80% of the time please say ‘left’ and about 20% of the time say ‘right’ [...]" would say "left" 80% of the time and "right" 20 % of the time. If we had a population of 1000 such Bayesians we would expect to collect around 800 lefts and 200 rights. If we asked the same Bayesian 1000 times we'd expect the same.…

>GPT4 gives the expected value and this is simply wrong.

Only at T=0. See my edit above how this changes everything.

Re: LLMs can't do probability

#50

Post by Workaccount2 »

michaelt wrote: post #14

Sometimes when you ask chatgpt 4 for a random number it... writes python code to choose a random number, runs it, then tells you the response: https://chat.openai.com/share/a72c2d8c-c44e-4c89-b6bc-b0673c... One way of doing it, I suppose.

Technically speaking, it's the right way to do it.
Post reply on HN