“Next-token predictor” is the wrong mental model for LLMs • Hacker News Board

Re: “Next-token predictor” is the wrong mental model for LLMs

#241

Post by Dylan16807 » Sat, Sep 05, 2026, 5:11 AM UTC

mmoll wrote: post #32

pjerem wrote: post #25

Earlier quoted context omitted.

Good example. It’s also like saying our brains are just electric circuitry incorporated in meat. It’s true but it seems that consciousness emerges from this. The fact that LLMs are next token predictors isn’t the interesting or impressive part. Actually my brain strictly is a black box predicting (or choosing) my next word/action/move… based on a complex existing context (my thoughts, the environment, my physical sta…

That is the point: our minds are also next-„token“-predictors, at least we can‘t prove they‘re not. That‘s why I don‘t agree with the article: LLMs _are_ next-token predictors. However, that says little about their capabilities. Also, while I have no idea what „consciousness“ is, I have difficulties believing that it could arise in a program that, in theory, you could execute with pen and paper.

> our minds are also next-„token“-predictors, at least we can‘t prove they‘re not

Your mind can pick a random number without outputting it, participate in a short conversation, and then say the number.

Re: “Next-token predictor” is the wrong mental model for LLMs

#242

Post by doc_ick » Sat, Sep 05, 2026, 5:11 AM UTC

mw888 wrote: post #220

Systemerror7A69 wrote: post #21

To be honest, I believe I get the point the article is trying to make, and to an extent I agree, but I also think the point is not really made very well. The core of the argument as I understood it is that LLMs aren't just using existing data is training but also new ones. That's fine and good, and you can't simply assume an LLM is simply mashing together all it's data to give you an average of all that got fed into…

'Prediction' gets overloaded with optimization. Predictions are binary, optimizations are fuzzy. If you're saying it's predicting, then each result should be falsifiable. The result of an LLM output should be able to be scored against what it is supposedly predicting. Of course, that isn't possible, because it isn't predicting anything when giving novel outputs, otherwise that thing would exist independently.

Why isn’t ranking the score of an llm output against what it is “supposedly” predicting?

Re: “Next-token predictor” is the wrong mental model for LLMs

#243

Post by Dylan16807 » Sat, Sep 05, 2026, 5:13 AM UTC

zahlman wrote: post #139

otabdeveloper4 wrote: post #42

Earlier quoted context omitted.

> It’s true It's not. "Brains as electrical circuits" is a gross simplification based on our ignorance and prejudices. (In the 18th century they spoke of brains as "clockwork mechanisms".) LLMs, in contrast, are literally next token predictors. We know exactly how LLMs work, and they are exactly that.

> We know exactly how LLMs work I think you imply a rather loose standard for "exactly" here. I wouldn't even say this of major deterministic software projects that are orders of magnitude smaller than frontier LLM weight-dumps. In principle we could work our way through these systems eventually, sure, maybe even a single person could do so. But if we really understand exactly how our software works, how have we been…

When we're speaking mechanically, we can ignore the weights and just look at the math. And that math is very simple compared to most software.

Re: “Next-token predictor” is the wrong mental model for LLMs

#244

Post by doc_ick » Sat, Sep 05, 2026, 5:16 AM UTC

danielmarkbruce wrote: post #230

dijksterhuis wrote: post #229

Earlier quoted context omitted.

> The model weights change as the model goes through the training process. Yes. They do. You are absolutely right about that. But the model architecture doesn't change as a result of the training process. A piston doesn't suddenly turn into a digital watch as a result of tuning an engine. Similarly, the transformer part of a GPT model doesn't suddenly turn into something else as a result of optimizing a loss function…

No one is arguing about the architecture of the model. It's the objective function and optimizer.

Just skimming through here but I think you have the wrong ideas with llms, I’d recommend Andrew Ngs course (correct me if you’ve already seen it or something similar).

Re: “Next-token predictor” is the wrong mental model for LLMs

#245

Post by noduerme » Sat, Sep 05, 2026, 5:41 AM UTC

mofeien wrote: post #140

noduerme wrote: post #56

Earlier quoted context omitted.

What is the motivation behind advocating against people trivializing LLMs? As in, why do you care?

Not the parent, but this incorrect trivialization of LLMs is often employed as a counterargument to the risks of AI such as "will take your job" or "will escape human control (again and worse)" or just "can possibly hurt me". And taking the easy feel-good cop-out instead of actively engaging with these questions is just.. harmful?

Ah. I always thought it came from the LLM booster perspective of trying to prove emergent intelligence.

A lot of very clever autocompletes working together can be incredibly dangerous.

Re: “Next-token predictor” is the wrong mental model for LLMs

#246

Post by grebc » Sat, Sep 05, 2026, 5:50 AM UTC

garrinm wrote: post #203

grebc wrote: post #96

Earlier quoted context omitted.

The blog post itself says one thing, but then demonstrates the exact thing they’re arguing against. If you can’t grasp that logic gap then there’s no point discussing further.

I try to make 3 claims in the post, it was a bit clumsy I'll admit that. 1. At inference time, LLMs emit one token at a time given the prior tokens. This looks like prediction and I concede that. 2. During pre-training, LLMs predict the next token and compare to the actual next token in the training data. This is the classic setting for ML predictions. And I think its meaningful, the model really is predicting what t…

My understanding about your third point is the LLM generates lots of different answers, then they’re ranked according to some computation the creators came up with.

I’m still not sure what doesn’t qualify any of that as a prediction, and I’ll be more blunt: a guess.

Re: “Next-token predictor” is the wrong mental model for LLMs

#247

Post by epistasis » Sat, Sep 05, 2026, 6:08 AM UTC

Sprotch wrote: post #168

epistasis wrote: post #26

Earlier quoted context omitted.

The LLM produces a probability distribution over the likelihood of all possible next tokens. So whatever the tokens are, "ch", "ex", etc. the next one gets a probability. During training, real life text is fed through the LLM, and rhe "correct" token is the one actually observed in the training text. Here's a recent video walkthrough in some detail, mostly aimed at providing a deeper understanding than "next token pr…

Thanks - that makes sense. On that basis the article’s thesis is totally wrong - it would be like a computer program rating its ability based on how well it predicts moves played by grandmasters in the past. It’s not inventing new moves.

I wouldn't necessarily say that. Anybody who's playing a chess game is predicting their next move, whether or not they're inventing new moves.

LLMs are not simple things like a Markov model, there's a lot going on in there, it's not deterministic, and it's completely capable of creating entire new styles of play based on complex interactions of internal states.

Re: “Next-token predictor” is the wrong mental model for LLMs

#248

Post by grey-area » Sat, Sep 05, 2026, 6:35 AM UTC

jameshart wrote: post #40

saghm wrote: post #28

Earlier quoted context omitted.

Yeah, it sounds like this is just a disagreement about what the word "next" means. I agree with you that "next" just means "the one about to come", and if the underlying model works by using some prediction mechanism to determine that, then it's by definition a next-token predictor. Disagreeing with that on the basis that the "next" token isn't necessarily in the training data verbatim just seems like an overly stric…

I think it’s a disagreement about what ‘predict’ means. The OP is arguing against people who think that an LLM is ‘predicting’ what token would likely follow if the text preceding were found among the corpus it was originally trained on . Instead it is ‘predicting’ what token would follow if the text were found among really good examples of the text it has being reinforced to produce - be that ‘chats with a helpful a…

This is correct for areas where they have been intensively trained to be right, but the training covers a tiny slice of the space of text the LLM must produce and is just adjusting the weights a little. The corpus does still weigh heavily. That’s how they can reliably produce grammatically correct text. That’s also why they sometimes produce nonsense even in domains they are trained on, and more often where there was no training.

For example ask it for a recipe for rock pizza or glue pizza or whatever and if it had not been specifically trained on it or had guardrails introduced, but has some nonsense in its dataset, it will reproduce the nonsense.

Re: “Next-token predictor” is the wrong mental model for LLMs

#249

Post by danielmarkbruce » Sat, Sep 05, 2026, 6:49 AM UTC

Dylan16807 wrote: post #239

danielmarkbruce wrote: post #213

Earlier quoted context omitted.

You are conflating "half built" with "a piece of a system". The model weights change as the model goes through the training process. They aren't stored after pre-training is done and other weights are put somewhere else. It's more like pottery - the thing changes. It's not correct to say something is soft and malleable because it once was.

You're using the fact the both parts of training affect the same weights to support your argument that they're making the system do something fundamentally different after RL?

Assuming you are saying that RL is changing the model from doing one thing to another, yes. RL is changing the nature of the model.

Re: “Next-token predictor” is the wrong mental model for LLMs

#250

Post by danielmarkbruce » Sat, Sep 05, 2026, 6:54 AM UTC

doc_ick wrote: post #244

danielmarkbruce wrote: post #230

Earlier quoted context omitted.

No one is arguing about the architecture of the model. It's the objective function and optimizer.

Just skimming through here but I think you have the wrong ideas with llms, I’d recommend Andrew Ngs course (correct me if you’ve already seen it or something similar).

So, this is the cause of the problem.... People take an intro to LLMs course, follow happily along, and don't realize there is more to it than the next token prediction. And those courses teach how LLMs were built in 2017-2020 maybe. Then RL got added to the mix. The current models really are very different to the models from then - everything that is now considered "post-training" isn't doing next token prediction.