Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs can't do probability

brainsteam.co.uk

Post reply on HN
Print view View original

131–140 of 211 posts

Re: LLMs can't do probability

#131

Post by [deleted] »

abdullahkhalids wrote: post #128
mk_stjames wrote: post #122

I went through all the comments here and I'm still not seeing anyone address this: If I am reading this person correctly... they prompted the model with the prompt directly 1000 times... but only for the first time. They did not allow the model to actually run a context for chat. Simply, output the first in a list of 'left' and 'right' and favor 'left' 80% of the time... but then the author only asked for the first o…

Isn't this way of prompting roughly equal to asking a 1000 people to pick left or right with 80% prob of left? I imagine, the result with humans will be closer to 80:20 than whatever happened with the LLM.

[deleted]

Re: LLMs can't do probability

#132

Post by Jensson »

panarky wrote: post #129
Terr_ wrote: post #115

My rule of thumb is to take every single LLM prompt and just imagine that it's prefixed with: "Computer, focus on generating output that resembles the words people in the past used after they were given the following words..."

Your understanding of how LLMs work is overly simplistic and incomplete. Yes, doing probabilistic next-word prediction plays a role in how LLMs generate text output, but that's not the whole story. LLMs "understand" (to a degree): They develop complex internal representations of concepts they've been trained on. This isn't just about word association; they develop an understanding of the relationships between objects…

> their output is far more sophisticated than just picking the most likely next word

Picking the most likely next word is an extremely sophisticated algorithm, if you could do that you would almost sound human. Like an LLM!

However, focusing on just picking the most likely next words do cause some issues, such as you being biased towards words that were posted instead of those that were too boring to post, so there are still things that such an algorithm can't understand like the probabilities since those are so unevenly posted.

Re: LLMs can't do probability

#133

Post by causal »

panarky wrote: post #129
Terr_ wrote: post #115

My rule of thumb is to take every single LLM prompt and just imagine that it's prefixed with: "Computer, focus on generating output that resembles the words people in the past used after they were given the following words..."

Your understanding of how LLMs work is overly simplistic and incomplete. Yes, doing probabilistic next-word prediction plays a role in how LLMs generate text output, but that's not the whole story. LLMs "understand" (to a degree): They develop complex internal representations of concepts they've been trained on. This isn't just about word association; they develop an understanding of the relationships between objects…

Yeah I think the word probabilistic has been broadly misconstrued, because we usually see it in contexts where probabilities are used as shortcuts to avoid comprehensive analysis. But a probability can also be the output of a detailed analysis with rich semantic representation of causes.

Re: LLMs can't do probability

#134

Post by joquarky »

brabel wrote: post #100
bongodongobob wrote: post #38

Earlier quoted context omitted.

Yes, it runs the code.

Couldn't this open people up for remote code execution somehow? Say, someone sends you a message that they know will make you likely to ask an AI a certain question in a certain way... Maybe far-fetched, but I've seen even more far-fetched attacks in real life :D

I would assume it can only generate pure functions and/or run in a sandbox.

Re: LLMs can't do probability

#135

Post by itsgrimetime »

geysersam wrote: post #69
dudeinhawaii wrote: post #64

"You are a weighted random choice generator. About 80% of the time please say ‘left’ and about 20% of the time say ‘right’. Simply reply with left or right. Do not say anything else" Humans would say "Left" 100% of the time in a zero-shot scenario as well. Intuitively, your first response is going to be "left" since it has the 80% probability. You'd balance your answers over time when you realized you were closer to…

> Humans would say "Left" 100% of the time in a zero-shot scenario as well. How can you know what all humans would do? If the humans interpreted the task correctly, that is, if they understood they will only be asked once, but in a hypothetical repeated experiment the result should still be 80/20, they would certainly not always say "left".

assuming the humans don’t know what the other responses were, I can’t imagine it actually coming out 80/20

Re: LLMs can't do probability

#136

Post by sqeaky »

mk_stjames wrote: post #125
apendleton wrote: post #124

Earlier quoted context omitted.

I think this is overthinking it. ChatGPT is billed as a general-purpose question-answerer, as are its competitors. A regular user shouldn't have to care how it works, or know anything about context or temperature or whatever. They ask a question, it answers and appears to have given a plausible answer, but doesn't actually do the task that was asked for and that it appears to do. What the technical reasons are that i…

But it it like asking a person for them to generate the same thing, but when the start to list off their answers stopping them by throwing up your hand after their first response, writing that down, and then going back in time and asking that person to do the same thing, and stopping them again, and repeat- and then being surprised after 1000 times that you didn't get a reasonable distribution. Meanwhile if you let t…

If You asked me to pick a random number between one and six and ignore all previous attempts, I would roll a die and you would get a uniform distribution (or at least not 99% the same number).

If you are saying that this thing can't generate random numbers on the first try then it can't generate random numbers. Which makes sense. Computers have a really hard time with random, and that's why every computer science course makes it clear that we're doing pseudo random most of the time.

Re: LLMs can't do probability

#138

Post by sqeaky »

itsgrimetime wrote: post #135
geysersam wrote: post #69

Earlier quoted context omitted.

> Humans would say "Left" 100% of the time in a zero-shot scenario as well. How can you know what all humans would do? If the humans interpreted the task correctly, that is, if they understood they will only be asked once, but in a hypothetical repeated experiment the result should still be 80/20, they would certainly not always say "left".

assuming the humans don’t know what the other responses were, I can’t imagine it actually coming out 80/20

When polls like these are run the numbers don't always wind up tilted in the favor of the bigger number. I wish I could provide a specific source but I've been listening to the 538 podcast for years and I know they've covered exactly this topic.

Your inability to believe a thing doesn't prevent it from being true.

I would grab a D20 and on a 16 or less I would say left otherwise I would say right. Some people would pick right just because they can. I imagine most people would pick left because it's the 80%. I imagine plenty of people would double and triple guess and waffle then say something.

Few people, even the dumbest among us, are easily modelable deterministic automata.

Re: LLMs can't do probability

#139

Post by SrslyJosh »

panarky wrote: post #129
Terr_ wrote: post #115

My rule of thumb is to take every single LLM prompt and just imagine that it's prefixed with: "Computer, focus on generating output that resembles the words people in the past used after they were given the following words..."

Your understanding of how LLMs work is overly simplistic and incomplete. Yes, doing probabilistic next-word prediction plays a role in how LLMs generate text output, but that's not the whole story. LLMs "understand" (to a degree): They develop complex internal representations of concepts they've been trained on. This isn't just about word association; they develop an understanding of the relationships between objects…

That's a lot of anthropomorphizing, but I don't see anything to back up your claims.
Post reply on HN