Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Next-token predictor” is the wrong mental model for LLMs

gmcgoldr.github.io

Post reply on HN
Print view View original

191–200 of 328 posts

Re: “Next-token predictor” is the wrong mental model for LLMs

#191

Post by wonnage »

mofeien wrote: post #107

Describing it as a "next-token predictor" in the sense that this would mean it's fundamentally limited to just a fraction of an inferential step is doubly wrong: 1. In order to select even the first word of a meaningful sentence, it already has to have structure and meaning of what follows captured somewhere inside, mostly in it's weights/activations or indexed by it's state vector. 2. What you see when you use an LL…

I think you are just litigating the philosophical debate of Bayesian vs frequentist interpretations of probability. Because the weights really are just probabilities.

Bayesians say that the probabilities represent strength of belief, implying some subjective knowledge or information. It is necessarily subjective in that it requires priors, i.e information the predictor knew before making the prediction. In other words, the LLM has priors from training and is predicting tokens using real knowledge

Frequentists would say that probabilities are simply objective facts - e.g we all agree that the physical property of temperature follows from any molecules matching a particular energy distribution. You’re not predicting anything, there’s just some outcomes that are happening at the expected rate. In other words, the LLM is a stochastic parrot/next token predictor

Re: “Next-token predictor” is the wrong mental model for LLMs

#192

Post by wonnage »

wat10000 wrote: post #147
gwerbin wrote: post #137

Earlier quoted context omitted.

But it literally is making a prediction based on the previous X tokens, it's just that X is huge and there is a proportionally huge number of parameters in the token generation function.

The discourse around this is annoying. A big group of people use "next-token predictor" to imply that LLMs aren't capable of anything interesting. Another big group of people opposes the use of "next-token predictor" because of that implication. But that fight isn't about the "predictor" language at all. The linked article makes a good point: a substantial chunk of the training does not consist of "here's a bunch of…

You can pour a bunch of beads into a pachinko board and end up with a normal distribution. Do the system “know” they need to go there?

Then you feed a bunch of tokens into a GPU and end up with a distribution of possible next tokens…

Re: “Next-token predictor” is the wrong mental model for LLMs

#193

Post by danielmarkbruce »

dijksterhuis wrote: post #187
danielmarkbruce wrote: post #184

Earlier quoted context omitted.

Read through the article and comments. You are talking solely about pre-training. I'm talking about post training. Respectfully, you are miles out of your depth. GPT-2 didn't use any reinforcement learning and is often given as a toy example. That release was 2019 and models now go through a various phases of training with different objective functions and optimizers.

From GP, i.e. the context for this local part of the thread > Autoregressive LLMs generate tokens one at a time, disputing this is just plain wrong. next-token prediction i.e. the bit built during pre-training. at no point in your reply to GP did you specify that you were referring to post-training. respectfully, it seems like this one is on you pal :shrug: > GPT-2 didn't use any reinforcement learning and is often g…

All modern LLMs that actually get used go through post-training. The finished product is something which has been through post training. So they are not next token prediction machines.

Re: “Next-token predictor” is the wrong mental model for LLMs

#194

Post by wonnage »

ozgung wrote: post #180

“LLMs are just X”, mostly with X = “next token predictors” is a common pattern to dismiss the power of AI with a very shallow understanding of how they really work. It’s not wrong, but because LLMs are generators, and generation is a kind of prediction. And current mainstream models are autoregressive, which means they generate things one by one in order. But these trivia doesn’t tell us anything interesting about ho…

The whole point of transformers is that you can take “a tiny ML model” and just scale it up 100000x and then it tells Zuckerberg what to bake with his kid

Re: “Next-token predictor” is the wrong mental model for LLMs

#195

Post by dijksterhuis »

danielmarkbruce wrote: post #193
dijksterhuis wrote: post #187

Earlier quoted context omitted.

From GP, i.e. the context for this local part of the thread > Autoregressive LLMs generate tokens one at a time, disputing this is just plain wrong. next-token prediction i.e. the bit built during pre-training. at no point in your reply to GP did you specify that you were referring to post-training. respectfully, it seems like this one is on you pal :shrug: > GPT-2 didn't use any reinforcement learning and is often g…

All modern LLMs that actually get used go through post-training. The finished product is something which has been through post training. So they are not next token prediction machines.

> The finished product is something which has been through post training.

again, the finished product wasn't what was discussed by GP, and you didn't clarify that you were switching to discussing RL (which is still probabilistic btw)

Re: “Next-token predictor” is the wrong mental model for LLMs

#196

Post by Dylan16807 »

danielmarkbruce wrote: post #167
wat10000 wrote: post #164

Earlier quoted context omitted.

Right. But that's never what the discussion is about. This is the first time I've seen someone say, "it's not just a next-token predictor because a bunch of the training isn't about predicting the next token." I have seen many, many people say "it's not just a next-token predictor because there's all this smart stuff going on inside the model." Even here, in the comments for an article that's all about the former, mo…

Yup, you are mostly right. I guess the people in my camp find the "it's just a next token predictor" stupid in that it's like saying "it's just a bunch of carbon and hydrogen", but it's also one of those things where people like to think they are clever because they think they are theoretically correct. But they aren't even that. So it's like double stupid. But the "next token predictor" part is at least technically…

It really is an important distinction, though. Being a next token predictor doesn't stop it from writing good sentences, but it does mean an LLM by itself can't play the number guessing game with you.

Re: “Next-token predictor” is the wrong mental model for LLMs

#197

Post by danielmarkbruce »

dijksterhuis wrote: post #189
danielmarkbruce wrote: post #182

Earlier quoted context omitted.

Nope. This isn't right.

> "it's not just a next-token predictor because a bunch of the training isn't about predicting the next token." clever procedures on top of the base transformer architecture. i used simplified words/phrases to summarise the same thing you two were saying (the intent being: here's a version that may be digestible when discussing with others). apparently that means i'm wrong though, no idea why because it seems you've…

They aren't predicting the next token. It's quite literally not a prediction.

Re: “Next-token predictor” is the wrong mental model for LLMs

#198

Post by danielmarkbruce »

garrinm wrote: post #150
angoragoats wrote: post #143

Earlier quoted context omitted.

> It doesn't check a "prediction" against what was actually "next" in data Literally no one here is claiming that it does. This is one of the many flaws in the article.

It does in pre training, but not in RL post training. And not at inference time. Reading over all these comments I get the feeling my mistake was not clearly delineating inference time and train time.

Your mistake was assuming people would be bothered to understand the details of how things work. Most people are lazy and don't know the details of how anything works.

Re: “Next-token predictor” is the wrong mental model for LLMs

#199

Post by garrinm »

drivebyhooting wrote: post #183

That chess analogy deeply confused me. Chess engines don’t compute win probabilities and choose the highest move. I don’t think a chess engine is an apt analogy at all. In a chess engine, there is a concrete search tree and although it emits one move at a time, it’s actually picking the entire branch (of course, with iterative deepening as the game progresses). There is no obvious place in transformer models where th…

Yes I understand the analogy was a bit loose. I'm comparing what happens at "inference time" in chess engines to what happens at train time in LLMs. In hindsight AlphaGo Zero was the perfect analogy, but I missed that opportunity.

The analogy with chess still works, but there's an extra step to think about. In both cases there is some kind of search over possible future trajectories. A chess engine explicitly searches branches of the game tree and evaluates which moves lead to good outcomes. In RL for an LLM, you sample rollouts, evaluate the resulting trajectories, and use those evaluations to update the policy.

The extra step with the LLM is that you don't keep doing that whole search at inference time. You use the rollouts to update the weights, so in some sense the useful information from that search gets compressed into the model.

But if you accept that the model is, in some loose sense, storing what it learned from those rollouts in its weights, then at inference time they are doing a similar job: taking some input state (prior tokens or a board position) and choosing the next action.

Re: “Next-token predictor” is the wrong mental model for LLMs

#200

Post by [deleted] »

danielmarkbruce wrote: post #197
dijksterhuis wrote: post #189

Earlier quoted context omitted.

> "it's not just a next-token predictor because a bunch of the training isn't about predicting the next token." clever procedures on top of the base transformer architecture. i used simplified words/phrases to summarise the same thing you two were saying (the intent being: here's a version that may be digestible when discussing with others). apparently that means i'm wrong though, no idea why because it seems you've…

They aren't predicting the next token. It's quite literally not a prediction.

[deleted]
Post reply on HN