“Next-token predictor” is the wrong mental model for LLMs • Hacker News Board

Re: “Next-token predictor” is the wrong mental model for LLMs

#181

Post by ltbarcly3 » Sat, Sep 05, 2026, 1:36 AM UTC

ozgung wrote: post #180

“LLMs are just X”, mostly with X = “next token predictors” is a common pattern to dismiss the power of AI with a very shallow understanding of how they really work. It’s not wrong, but because LLMs are generators, and generation is a kind of prediction. And current mainstream models are autoregressive, which means they generate things one by one in order. But these trivia doesn’t tell us anything interesting about ho…

It's not simple to do next token prediction. That IS what is going on. You want something 'deep'. Deep things are often very not complicated.

The deep realization is that if you can predict the next token well enough, you can do things like this:

. And it turned out the killer was

And if it's really good at predicting the next token, it has to understand the novel and the clues, which means understanding the context and the language and human norms and innuendo and story telling, and tropes, and red herrings, and predict who the killer was.

I think you want it to be something more complicated. It's literally not. It just turns out predicting the next token is equivalent to a universal compression algorithm, which is a form of general intelligence. And we have almost unlimited 'labeled' data to train autocomplete.

Re: “Next-token predictor” is the wrong mental model for LLMs

#182

Post by danielmarkbruce » Sat, Sep 05, 2026, 1:39 AM UTC

dijksterhuis wrote: post #176

danielmarkbruce wrote: post #167

Earlier quoted context omitted.

Yup, you are mostly right. I guess the people in my camp find the "it's just a next token predictor" stupid in that it's like saying "it's just a bunch of carbon and hydrogen", but it's also one of those things where people like to think they are clever because they think they are theoretically correct. But they aren't even that. So it's like double stupid. But the "next token predictor" part is at least technically…

they are cleverly arranged / configured next-most-likely-token predictors, possibly with some clever procedures / attachments on top.

Nope. This isn't right.

Re: “Next-token predictor” is the wrong mental model for LLMs

#183

Post by drivebyhooting » Sat, Sep 05, 2026, 1:39 AM UTC

That chess analogy deeply confused me. Chess engines don’t compute win probabilities and choose the highest move.

I don’t think a chess engine is an apt analogy at all. In a chess engine, there is a concrete search tree and although it emits one move at a time, it’s actually picking the entire branch (of course, with iterative deepening as the game progresses).

There is no obvious place in transformer models where the entire trace was already computed prior to a single token being chosen. It’s possible, maybe even likely, that the whole trace exists internally as activations. Multi token prediction and diffusion adapters point to that being the case. But to my knowledge no explanation has been given for where in the model the future plan is stored.

Re: “Next-token predictor” is the wrong mental model for LLMs

#184

Post by danielmarkbruce » Sat, Sep 05, 2026, 1:40 AM UTC

dijksterhuis wrote: post #169

danielmarkbruce wrote: post #154

Earlier quoted context omitted.

Emitting and predicting are different things though. Prediction implies there is some "truth" or event or something that you can test against. Prediction implies the model just learns from existing text, and optimizes to predict the next token in training data. That's just not true.

prediction is a very specific term of art in the field of machine learning. generally speaking, machine learning models like LLMs are based on probability; performing a statistical prediction of the likely y given some input x Probability(y | x) that's why we refer to outputs as a prediction. it is likelihoods and stuff. the output is never definitely correct as we're not dealing with heuristic processes. > Predictio…

Read through the article and comments. You are talking solely about pre-training. I'm talking about post training.

Respectfully, you are miles out of your depth. GPT-2 didn't use any reinforcement learning and is often given as a toy example. That release was 2019 and models now go through a various phases of training with different objective functions and optimizers.

Re: “Next-token predictor” is the wrong mental model for LLMs

#185

Post by danielmarkbruce » Sat, Sep 05, 2026, 1:42 AM UTC

jvanderbot wrote: post #171

danielmarkbruce wrote: post #81

Earlier quoted context omitted.

There isn't a truth to test against. If I predict the next word in a sequence is "sat", we can check against the sequence. If I predict the roll of a die will be 4, we can check against it. Whether i give 100% or give a probabilistic prediction, we can check against the truth. If I choose a specific move in chess, it's a choice. It's not a prediction. I might get a score 40 moves later given my choice, but I'm not pr…

Oh, that's somewhat philosophical. The prediction here is a mechanical thing, just random sampling along a pre-calibrated mapping of options given the tokens and their importance weights. That sampling can be called prediction if you use it that way, or it can be called thinking if you are into the psychosis too deeply. Or it can be called a CLI tool command if you're in an agentic framework

No, it's not philsophical. Because if you optimize to predict, you are doing something different to optimizing for a reward. It's a different process - different objective function, different optimization, different set up.

Re: “Next-token predictor” is the wrong mental model for LLMs

#186

Post by [deleted] » Sat, Sep 05, 2026, 1:44 AM UTC

danielmarkbruce wrote: post #184

dijksterhuis wrote: post #169

Earlier quoted context omitted.

prediction is a very specific term of art in the field of machine learning. generally speaking, machine learning models like LLMs are based on probability; performing a statistical prediction of the likely y given some input x Probability(y | x) that's why we refer to outputs as a prediction. it is likelihoods and stuff. the output is never definitely correct as we're not dealing with heuristic processes. > Predictio…

Read through the article and comments. You are talking solely about pre-training. I'm talking about post training. Respectfully, you are miles out of your depth. GPT-2 didn't use any reinforcement learning and is often given as a toy example. That release was 2019 and models now go through a various phases of training with different objective functions and optimizers.

[deleted]

Re: “Next-token predictor” is the wrong mental model for LLMs

#187

Post by dijksterhuis » Sat, Sep 05, 2026, 1:49 AM UTC

danielmarkbruce wrote: post #184

dijksterhuis wrote: post #169

Earlier quoted context omitted.

prediction is a very specific term of art in the field of machine learning. generally speaking, machine learning models like LLMs are based on probability; performing a statistical prediction of the likely y given some input x Probability(y | x) that's why we refer to outputs as a prediction. it is likelihoods and stuff. the output is never definitely correct as we're not dealing with heuristic processes. > Predictio…

Read through the article and comments. You are talking solely about pre-training. I'm talking about post training. Respectfully, you are miles out of your depth. GPT-2 didn't use any reinforcement learning and is often given as a toy example. That release was 2019 and models now go through a various phases of training with different objective functions and optimizers.

From GP, i.e. the context for this local part of the thread

> Autoregressive LLMs generate tokens one at a time, disputing this is just plain wrong.

next-token prediction i.e. the bit built during pre-training.

at no point in your reply to GP did you specify that you were referring to post-training. respectfully, it seems like this one is on you pal :shrug:

> GPT-2 didn't use any reinforcement learning and is often given as a toy example. That release was 2019 and models now go through a various phases of training with different objective functions and optimizers.

yeah. so? the toy example works for pre-training. see above.

Re: “Next-token predictor” is the wrong mental model for LLMs

#188

Post by bloppe » Sat, Sep 05, 2026, 1:54 AM UTC

gjm11 wrote: post #29

mort96 wrote: post #20

Earlier quoted context omitted.

At the same time, it ... is literally a next token predictor. Like that's what it is. The input is a sequence of tokens. The output is a probability distribution of next tokens.

It is. And human beings are bags of chemicals. But for many purposes you will not find it helpful to think of human beings as bags of chemicals, and for many purposes you will not find it helpful to think of LLMs as next-token predictors.

I think a better simplistic analogy would be that humans are progeny-maximizers. Optimization problems can give rise to all sorts of interesting behaviors but the simplistic perspective is also useful and interesting in both cases

Re: “Next-token predictor” is the wrong mental model for LLMs

#189

Post by dijksterhuis » Sat, Sep 05, 2026, 2:06 AM UTC

danielmarkbruce wrote: post #182

dijksterhuis wrote: post #176

Earlier quoted context omitted.

they are cleverly arranged / configured next-most-likely-token predictors, possibly with some clever procedures / attachments on top.

Nope. This isn't right.

> "it's not just a next-token predictor because a bunch of the training isn't about predicting the next token."

clever procedures on top of the base transformer architecture.

i used simplified words/phrases to summarise the same thing you two were saying (the intent being: here's a version that may be digestible when discussing with others).

apparently that means i'm wrong though, no idea why because it seems you've decided to be dismissive rather than constructively elaborate on why this simplified and digestible version might be wrong :shrug:

Re: “Next-token predictor” is the wrong mental model for LLMs

#190

Post by randyrand » Sat, Sep 05, 2026, 2:06 AM UTC

emtel wrote: post #174

I would say that it is not even wrong. You say it’s a next token predictor. I say it isn’t. What observable behavior of the system can settle our dispute? I fail to see how any possible output could cause either of us to change our mind.

What comes after “the dog”?

Does it complete the sentence?

If not then it’s not a next token predictor. Or at least not a good one.