Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Next-token predictor” is the wrong mental model for LLMs

gmcgoldr.github.io

Post reply on HN
Print view View original

171–180 of 329 posts

Re: “Next-token predictor” is the wrong mental model for LLMs

#171

Post by jvanderbot »

danielmarkbruce wrote: post #81
jvanderbot wrote: post #65

Earlier quoted context omitted.

In what sense is the model not predicting?

There isn't a truth to test against. If I predict the next word in a sequence is "sat", we can check against the sequence. If I predict the roll of a die will be 4, we can check against it. Whether i give 100% or give a probabilistic prediction, we can check against the truth. If I choose a specific move in chess, it's a choice. It's not a prediction. I might get a score 40 moves later given my choice, but I'm not pr…

Oh, that's somewhat philosophical. The prediction here is a mechanical thing, just random sampling along a pre-calibrated mapping of options given the tokens and their importance weights. That sampling can be called prediction if you use it that way, or it can be called thinking if you are into the psychosis too deeply. Or it can be called a CLI tool command if you're in an agentic framework

Re: “Next-token predictor” is the wrong mental model for LLMs

#172

Post by garrinm »

stanleykm wrote: post #5

> Calling the second system a “next-move predictor” would be strange. It is not trying to predict what move appeared next in a dataset. It is trying to choose a move that wins. i dont understand the distinction here. does working backwards from a set of win states instead of working forwards from the current state somehow change whether it’s a prediction or not?

In the article I made 3 claims, and I agree it was a bit clumsy.

1st I say that "working forwards" in the sense of outputting one token at a time could be some form of prediction, I don't argue against that. This is what LLMs do at inference time.

2nd I say that to me what really constitutes a prediction is the pre-training. Here it's the classic setting for the word prediction in ML. The model outputs a prediction of the ground truth label: the next token.

3rd I argue that in RL there is no ground truth next token, so prediction doesn't apply here anymore.

Back to your question then: you're asking points 3 and 1 are different. Working backwards from a set of win states is basically what RL does in training. Working forward from the current state is what inference does. To me there is a distinction worth thinking about. First between the mechanism at inference time and at train time. Then between what happens in pre-training vs. RL post training.

Re: “Next-token predictor” is the wrong mental model for LLMs

#175

Post by arjie »

They are indeed "just next-token predictors". But rockets to orbit are also "just forward and upward goers". And Steph Curry is "just next three-pointer scorer" and so on. And as the article points out, chess engines are also "just next-move makers". A flawless oracular trading system is "just next trade maker". It turns out that "next n doer" is a true model that illuminates little.

Re: “Next-token predictor” is the wrong mental model for LLMs

#176

Post by dijksterhuis »

danielmarkbruce wrote: post #167
wat10000 wrote: post #164

Earlier quoted context omitted.

Right. But that's never what the discussion is about. This is the first time I've seen someone say, "it's not just a next-token predictor because a bunch of the training isn't about predicting the next token." I have seen many, many people say "it's not just a next-token predictor because there's all this smart stuff going on inside the model." Even here, in the comments for an article that's all about the former, mo…

Yup, you are mostly right. I guess the people in my camp find the "it's just a next token predictor" stupid in that it's like saying "it's just a bunch of carbon and hydrogen", but it's also one of those things where people like to think they are clever because they think they are theoretically correct. But they aren't even that. So it's like double stupid. But the "next token predictor" part is at least technically…

they are cleverly arranged / configured next-most-likely-token predictors, possibly with some clever procedures / attachments on top.

Re: “Next-token predictor” is the wrong mental model for LLMs

#178

Post by tetrisgm »

emtel wrote: post #174

I would say that it is not even wrong. You say it’s a next token predictor. I say it isn’t. What observable behavior of the system can settle our dispute? I fail to see how any possible output could cause either of us to change our mind.

That's perfectly put. The value of words and what we assign to them, let alone how we assign meaning, or the interpretation of what the other person says, is completely arbitrary. (in practice).

Re: “Next-token predictor” is the wrong mental model for LLMs

#179

Post by roenxi »

Systemerror7A69 wrote: post #21

To be honest, I believe I get the point the article is trying to make, and to an extent I agree, but I also think the point is not really made very well. The core of the argument as I understood it is that LLMs aren't just using existing data is training but also new ones. That's fine and good, and you can't simply assume an LLM is simply mashing together all it's data to give you an average of all that got fed into…

> It's just not predicting based on it's training data, but predicting based on RLVR & more, trying to get to the optimal solution ( as much as the solutions CAN be optimal)

It is predicting based on a model. In many cases we can download the model off hugging face. The model is conditioned by all sorts of things. Training data, post-training, coincidence, prompt inputs, runtime data available from whatever means.

> but at least I would still call it a "next token predictor"

We can call any prediction system a next token predictor. If you watch over the shoulder of a human writing a HN comment you are almost certain to see them generating a linear string of tokens. That is what keyboards do. It is impossible to generate text without being equivalent to a next token predictor.

Re: “Next-token predictor” is the wrong mental model for LLMs

#180

Post by ozgung »

“LLMs are just X”, mostly with X = “next token predictors” is a common pattern to dismiss the power of AI with a very shallow understanding of how they really work.

It’s not wrong, but because LLMs are generators, and generation is a kind of prediction. And current mainstream models are autoregressive, which means they generate things one by one in order. But these trivia doesn’t tell us anything interesting about how they work or their limits.

It’s like saying a Boeing 777 is just a rotating machine, and it flies by just rotating some fins. Well yes, but no. With that level of simplification we’ve just ignored 150 tons of advanced engineering and physics. Similarly with token generator simplification we ignore a few trillion parameter Transformer. That transformer is more complex than a Boeing 777, and we don’t really know how it works.

A tiny ML model can do “next token prediction”. This is not as simple as that.

Post reply on HN