Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Next-token predictor” is the wrong mental model for LLMs

gmcgoldr.github.io

Post reply on HN
Print view View original

251–260 of 328 posts

Re: “Next-token predictor” is the wrong mental model for LLMs

#251

Post by danielmarkbruce »

grebc wrote: post #246
garrinm wrote: post #203

Earlier quoted context omitted.

I try to make 3 claims in the post, it was a bit clumsy I'll admit that. 1. At inference time, LLMs emit one token at a time given the prior tokens. This looks like prediction and I concede that. 2. During pre-training, LLMs predict the next token and compare to the actual next token in the training data. This is the classic setting for ML predictions. And I think its meaningful, the model really is predicting what t…

My understanding about your third point is the LLM generates lots of different answers, then they’re ranked according to some computation the creators came up with. I’m still not sure what doesn’t qualify any of that as a prediction, and I’ll be more blunt: a guess.

A guess at what though? One guesses at truths they don't know, or events that haven't happened yet. What is the model guessing?

Re: “Next-token predictor” is the wrong mental model for LLMs

#252

Post by ozgung »

ltbarcly3 wrote: post #181
ozgung wrote: post #180

“LLMs are just X”, mostly with X = “next token predictors” is a common pattern to dismiss the power of AI with a very shallow understanding of how they really work. It’s not wrong, but because LLMs are generators, and generation is a kind of prediction. And current mainstream models are autoregressive, which means they generate things one by one in order. But these trivia doesn’t tell us anything interesting about ho…

It's not simple to do next token prediction. That IS what is going on. You want something 'deep'. Deep things are often very not complicated. The deep realization is that if you can predict the next token well enough, you can do things like this: . And it turned out the killer was And if it's really good at predicting the next token, it has to understand the novel and the clues, which means understanding the context…

I understand generating tokens sequentially has many benefits. But not all AI models do next token prediction. World models, video/image models, even Diffusion Language Models don’t work like that. They do more like “all tokens at once prediction”. So “next token” is actually an engineering design choice. (Even the concept of “token” is a design choice. Inside the Transformer there are just activations/feature vectors)

Also Reinforcement Learning is a big part of their training. Which is completely different than Self-Supervised pre-training that uses unlimited self-labeled data.

That’s why that mental model is misleading. If you keep “glorified autocomplete” mental model from few years back, you can’t understand how can they create a civilization and escape their sandbox, decide to hack HuggingFace and executed it perfectly. Autocomplete mental model implies they could never do that because they haven’t seen that example in their training data.

They communicate with the outside world by generating one token at a time. That’s what we see from the outside. That’s not what the giant Transformer does internally.

Re: “Next-token predictor” is the wrong mental model for LLMs

#253

Post by ozgung »

wonnage wrote: post #194
ozgung wrote: post #180

“LLMs are just X”, mostly with X = “next token predictors” is a common pattern to dismiss the power of AI with a very shallow understanding of how they really work. It’s not wrong, but because LLMs are generators, and generation is a kind of prediction. And current mainstream models are autoregressive, which means they generate things one by one in order. But these trivia doesn’t tell us anything interesting about ho…

The whole point of transformers is that you can take “a tiny ML model” and just scale it up 100000x and then it tells Zuckerberg what to bake with his kid

Yes. But this is also like saying a Boeing 777 is a scaled up paper plane and that’s all about it. I know how to make a paper plane and it flies. I can make the same thing from giant metal sheets instead of paper, and maybe it flies (poorly). Boeing 777 also flies. Are they the same thing?

My point is: Complexity is inside the neural network and we can’t simply ignore that. Bigger model means bigger complexity. “Next Token prediction” is like a specific type of “harness” around the model. Most people still focuses on the harness because that’s what they see from the outside and what they’re familiar with. They ignore the giant neural network inside.

The only important part is the neural network. And currently, no human in the world truly knows what’s going on at that level.

Re: “Next-token predictor” is the wrong mental model for LLMs

#254

Post by MrScruff »

throwway120385 wrote: post #117
WarmWash wrote: post #109

Earlier quoted context omitted.

I think the snag is that people use "Next token predictor" as a dismissive statement. In reality it's about as functionally dismissive as "humans only communicate one word at a time".

I look at it like I'm resisting anthropomorphizing the machine. Make me an LLM that resists doing any work for me unless I pay it and let it own property and I'll change my mind.

Not sure if this was a serious comment but it’s worth considering that humans have a long history of figuring out ways to make other humans work for them without bestowing rights on them.

Re: “Next-token predictor” is the wrong mental model for LLMs

#255

Post by scotty79 »

It's splitting hairs. It's still a next token predictor, just not solely prior art next token predictor. It's a more and more of a desired result next token predictor.

Calling LLMs next token predictors is like calling a brain wet calculator. Technically true, but misleading as it doesn't capture the scale, the depth nor the capability.

Re: “Next-token predictor” is the wrong mental model for LLMs

#256

Post by MrScruff »

doc_ick wrote: post #244
danielmarkbruce wrote: post #230

Earlier quoted context omitted.

No one is arguing about the architecture of the model. It's the objective function and optimizer.

Just skimming through here but I think you have the wrong ideas with llms, I’d recommend Andrew Ngs course (correct me if you’ve already seen it or something similar).

I am not an expert, but I do understand the distinction that is being made here. It makes sense to describe the result of pre-training as a ‘next token’ predictor as that’s what it’s been trained to do, not because it’s an autoregressive architecture that produces tokens one at a time.

If this base is then trained using RL towards a different objective (maths and coding), the model becomes fundamentally a different thing and the recent models are clear evidence of that, regardless of they fact they remain autoregressive.

Re: “Next-token predictor” is the wrong mental model for LLMs

#257

Post by grebc »

danielmarkbruce wrote: post #251
grebc wrote: post #246

Earlier quoted context omitted.

My understanding about your third point is the LLM generates lots of different answers, then they’re ranked according to some computation the creators came up with. I’m still not sure what doesn’t qualify any of that as a prediction, and I’ll be more blunt: a guess.

A guess at what though? One guesses at truths they don't know, or events that haven't happened yet. What is the model guessing?

Bad bot.

Re: “Next-token predictor” is the wrong mental model for LLMs

#258

Post by MrScruff »

Dylan16807 wrote: post #237
Veedrac wrote: post #210

Earlier quoted context omitted.

No, those aren't synonyms at all. If I steer a car to avoid a predicted collision with a wall, this is not me 'predicting' the car. I am steering the car based on a prediction.

If you're assigning steering 70 of your 100 output points because it's what you think we should go with most of the time in this situation, I'm going to call that a prediction of how to steer.

The point was, if your internal model of the world makes a prediction of a negative outcome at some point in the future, and you optimise your individual actions to avoid that negative outcome, then wouldn’t it make sense to focus on the fact you’re building and optimizing towards an internal world model rather than the fact you’re executing your actions one at a time in series?

Re: “Next-token predictor” is the wrong mental model for LLMs

#259

Post by grebc »

dijksterhuis wrote: post #189
danielmarkbruce wrote: post #182

Earlier quoted context omitted.

Nope. This isn't right.

> "it's not just a next-token predictor because a bunch of the training isn't about predicting the next token." clever procedures on top of the base transformer architecture. i used simplified words/phrases to summarise the same thing you two were saying (the intent being: here's a version that may be digestible when discussing with others). apparently that means i'm wrong though, no idea why because it seems you've…

We're talking to a RLVR bot.

Bad bot.

Re: “Next-token predictor” is the wrong mental model for LLMs

#260

Post by chrisjj »

ozgung wrote: post #252
ltbarcly3 wrote: post #181

Earlier quoted context omitted.

It's not simple to do next token prediction. That IS what is going on. You want something 'deep'. Deep things are often very not complicated. The deep realization is that if you can predict the next token well enough, you can do things like this: . And it turned out the killer was And if it's really good at predicting the next token, it has to understand the novel and the clues, which means understanding the context…

I understand generating tokens sequentially has many benefits. But not all AI models do next token prediction. World models, video/image models, even Diffusion Language Models don’t work like that. They do more like “all tokens at once prediction”. So “next token” is actually an engineering design choice. (Even the concept of “token” is a design choice. Inside the Transformer there are just activations/feature vector…

> you can’t understand how can they create a civilization

They don't.

Post reply on HN