Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Next-token predictor” is the wrong mental model for LLMs

gmcgoldr.github.io

Post reply on HN
Print view View original

151–160 of 335 posts

Re: “Next-token predictor” is the wrong mental model for LLMs

#151

Post by danielmarkbruce »

gwerbin wrote: post #137
Planktonne wrote: post #134

Earlier quoted context omitted.

> Describing it as a "next-token predictor" in the sense that this would mean it's fundamentally limited to just a fraction of an inferential step I don't think anyone is doing that though; we know LLMs are not simple Markov chains, and that the prediction they make is based on more than the previous X words. It's not minimising to describe even a complex prediction process as prediction.

But it literally is making a prediction based on the previous X tokens, it's just that X is huge and there is a proportionally huge number of parameters in the token generation function.

If you are going to say "literally", then what is your literal definition for the word "prediction" ?

Re: “Next-token predictor” is the wrong mental model for LLMs

#152

Post by andai »

>Calling the second system a “next-move predictor” would be strange. It is not trying to predict what move appeared next in a dataset. It is trying to choose a move that wins.

Sounds like the next 1000 years depend on how carefully we define "winning".

Re: “Next-token predictor” is the wrong mental model for LLMs

#153

Post by [deleted] »

wat10000 wrote: post #147
gwerbin wrote: post #137

Earlier quoted context omitted.

But it literally is making a prediction based on the previous X tokens, it's just that X is huge and there is a proportionally huge number of parameters in the token generation function.

The discourse around this is annoying. A big group of people use "next-token predictor" to imply that LLMs aren't capable of anything interesting. Another big group of people opposes the use of "next-token predictor" because of that implication. But that fight isn't about the "predictor" language at all. The linked article makes a good point: a substantial chunk of the training does not consist of "here's a bunch of…

[deleted]

Re: “Next-token predictor” is the wrong mental model for LLMs

#154

Post by danielmarkbruce »

Kranar wrote: post #106
Dlemlo wrote: post #77

Earlier quoted context omitted.

Blog articles from Anthropic and others show that this is not true. A LLM already knows more tokens than the current one. It was mentioned in a blog post about how a LLM is doing haikus and co. There are also structures in an LLM which allows it to 'estimate' numbers to a certain degree and doing other things.

You're misinterpreting these articles. Autoregressive LLMs generate tokens one at a time, disputing this is just plain wrong. What is true, however, is that in order to generate the next token autoregressive LLMs produce internal/hidden state about future tokens far past the next token so that it's not like the entire machinery of the algorithm deprives itself of representing where the sentence/text is headed. So "em…

Emitting and predicting are different things though. Prediction implies there is some "truth" or event or something that you can test against. Prediction implies the model just learns from existing text, and optimizes to predict the next token in training data. That's just not true.

Re: “Next-token predictor” is the wrong mental model for LLMs

#155

Post by danielmarkbruce »

Planktonne wrote: post #132
garrinm wrote: post #122

Earlier quoted context omitted.

The distinction I perhaps didn’t make clearly enough is that I’m not really debating the concept of prediction at inference time, although, as I pointed out elsewhere, I think that’s the less interesting interpretation of what “prediction” means. What’s more interesting to me is its application at training time. In reinforcement learning, there is no ground-truth next token to predict. So if you’re comfortable callin…

I think you're trying to limit the meaning of both 'next' and 'prediction' in ways that don't reflect usage and that--if adopted--would severely limit our ability to discuss and evolve what LLMs are actually doing. There's nothing inherent in either word that forces such a limit; predicting based on what will lead to success as measured by [reward function] is still a prediction.

It's not a prediction of the next move though, and that is the point. It's a prediction of what will happen if you make that move.

So, it's not a next move predictor. It's a game result predictor.

Re: “Next-token predictor” is the wrong mental model for LLMs

#157

Post by amluto »

gwerbin wrote: post #137
Planktonne wrote: post #134

Earlier quoted context omitted.

> Describing it as a "next-token predictor" in the sense that this would mean it's fundamentally limited to just a fraction of an inferential step I don't think anyone is doing that though; we know LLMs are not simple Markov chains, and that the prediction they make is based on more than the previous X words. It's not minimising to describe even a complex prediction process as prediction.

But it literally is making a prediction based on the previous X tokens, it's just that X is huge and there is a proportionally huge number of parameters in the token generation function.

It has nothing to do with X being huge. In fact X might be quite small.

Re: “Next-token predictor” is the wrong mental model for LLMs

#158

Post by amluto »

zahlman wrote: post #115
mofeien wrote: post #107

Describing it as a "next-token predictor" in the sense that this would mean it's fundamentally limited to just a fraction of an inferential step is doubly wrong: 1. In order to select even the first word of a meaningful sentence, it already has to have structure and meaning of what follows captured somewhere inside, mostly in it's weights/activations or indexed by it's state vector. 2. What you see when you use an LL…

Even so, one might wonder why we don't try making systems that take different approaches. For example, after a traditional first pass of output, they could do sliding-window "optimizations" considering each token in the context of tokens both before and after , and possibly replace words or phrases in-place. For example, I've noticed quite a few cases recently of LLMs outputting "but" where "and" would make more sens…

Look up diffusion models.

Re: “Next-token predictor” is the wrong mental model for LLMs

#159

Post by danielmarkbruce »

wat10000 wrote: post #147
gwerbin wrote: post #137

Earlier quoted context omitted.

But it literally is making a prediction based on the previous X tokens, it's just that X is huge and there is a proportionally huge number of parameters in the token generation function.

The discourse around this is annoying. A big group of people use "next-token predictor" to imply that LLMs aren't capable of anything interesting. Another big group of people opposes the use of "next-token predictor" because of that implication. But that fight isn't about the "predictor" language at all. The linked article makes a good point: a substantial chunk of the training does not consist of "here's a bunch of…

The fight is about the predictor language in some cases. Because it's only a trivial difference to those who don't understand the details of how these things are made. In pre-training the model really is trained to predict the next token. What is being emitted by the model is, by structure, by training and by optimization, a prediction of the very next token.

What is emitted by a model during RLHF and RLVR is not, by structure, training or optimization, a prediction of the next token.

Re: “Next-token predictor” is the wrong mental model for LLMs

#160

Post by grebc »

danielmarkbruce wrote: post #149
grebc wrote: post #130

Earlier quoted context omitted.

Yes, no logic is necessary for LLM adherents we're all finding out. Carry on good soldier.

If you haven't built one, and don't understand how they work, why comment?

You don't need to build a car to understand one.

That you tie yourself up in knots of fancy acronyms instead of plain words and that your argument boils down to semantics of the word prediction, it's pretty clear what is up brother.

Post reply on HN