[deleted]
“Next-token predictor” is the wrong mental model for LLMs
231–240 of 328 posts
Re: “Next-token predictor” is the wrong mental model for LLMs
#232Earlier quoted context omitted.
It really is an important distinction, though. Being a next token predictor doesn't stop it from writing good sentences, but it does mean an LLM by itself can't play the number guessing game with you.
This is pedantic, but, actually RL has improved the quality of sentence construction in LLMs quite dramatically... And once you do some RL on that model, it aint a next token prediction machine any longer.
That's just training, isn't it? (Even if the term is "post-training") It doesn't change how inference works.
My understanding is it usually strengthens the "thinking ahead" part of inference, but that part was already there, and it's still at the end of the day picking one token and then purging internal state in a way that can only partially be recovered from.
Re: “Next-token predictor” is the wrong mental model for LLMs
#233Earlier quoted context omitted.
This is pedantic, but, actually RL has improved the quality of sentence construction in LLMs quite dramatically... And once you do some RL on that model, it aint a next token prediction machine any longer.
That's just training, isn't it? (Even if the term is "post-training") It doesn't change how inference works. My understanding is it usually strengthens the "thinking ahead" part of inference, but that part was already there, and it's still at the end of the day picking one token and then purging internal state in a way that can only partially be recovered from.
How inference works is you do a bunch of math and token weights come out, and then you pick one based on some sampling policy. “Prediction” is really about how they’re trained than how they run afterwards.
Re: “Next-token predictor” is the wrong mental model for LLMs
#234Earlier quoted context omitted.
That's just training, isn't it? (Even if the term is "post-training") It doesn't change how inference works. My understanding is it usually strengthens the "thinking ahead" part of inference, but that part was already there, and it's still at the end of the day picking one token and then purging internal state in a way that can only partially be recovered from.
How inference works is you do a bunch of math and token weights come out, and then you pick one based on some sampling policy. “Prediction” is really about how they’re trained than how they run afterwards.
[deleted]
Re: “Next-token predictor” is the wrong mental model for LLMs
#235Earlier quoted context omitted.
This is pedantic, but, actually RL has improved the quality of sentence construction in LLMs quite dramatically... And once you do some RL on that model, it aint a next token prediction machine any longer.
That's just training, isn't it? (Even if the term is "post-training") It doesn't change how inference works. My understanding is it usually strengthens the "thinking ahead" part of inference, but that part was already there, and it's still at the end of the day picking one token and then purging internal state in a way that can only partially be recovered from.
The discussion is basically: what is a model trying to do?
One may reasonably assert it isn't trying to do anything. But, in practice, if you give it an objective function and optimize it, the model is basically trained to "do" something. So what is it trained to "do"? During pre training it is trained to produce a distribution which is a prediction of the next token in it's training data samples. During RLVR and RLHF, it is trained to produce a distribution of tokens that will maximize a scoring function over many steps - not just the next step. The fact that it produces a distribution of potential choices for the next step doesn't mean the next step is a prediction. It's more of a "strategy" or "probabilistic path choice". The word used in RL is a "policy". It's a decent word to describe what the model is.
So, modern LLMs are trying to produce a good sequence of tokens. They are "good token sequence producer machines". Not "next token prediction machines". Pre RLHF (in practice, go back to pre chatgpt) they really were "next token prediction machines".
Re: “Next-token predictor” is the wrong mental model for LLMs
#236I'm not a neuroscientist, Our brains also rely heavily on prediction, using prior context and experience to anticipate what comes next, Also, people those who talk about the deterministic nature, LLM doesn't need to be deterministic, Human reasoning and behaviour aren't perfectly repeatable either. its the harness and the tools that use LLM should be deterministic, while the LLM can remain the probabilistic reasoning component.
From my own experience, I worked as a photogrammetrist at a university, where we used to build terrain models of very dense forest areas. When there was a steep hill or sudden change in terrain, my brain could see it either as a convex hill or as a concave depression. It often depended on how I was thinking about it. The same image could suddenly look completely different even though nothing in the image had changed. The only way to confirm it was by looking at the surrounding terrain and using our experience to understand what was actually there.
Re: “Next-token predictor” is the wrong mental model for LLMs
#237Earlier quoted context omitted.
> _determining_ the next token based on a prediction Also known as predicting.
No, those aren't synonyms at all. If I steer a car to avoid a predicted collision with a wall, this is not me 'predicting' the car. I am steering the car based on a prediction.
If you're assigning steering 70 of your 100 output points because it's what you think we should go with most of the time in this situation, I'm going to call that a prediction of how to steer.
Re: “Next-token predictor” is the wrong mental model for LLMs
#238Earlier quoted context omitted.
> It’s true It's not. "Brains as electrical circuits" is a gross simplification based on our ignorance and prejudices. (In the 18th century they spoke of brains as "clockwork mechanisms".) LLMs, in contrast, are literally next token predictors. We know exactly how LLMs work, and they are exactly that.
> We know exactly how LLMs work I think you imply a rather loose standard for "exactly" here. I wouldn't even say this of major deterministic software projects that are orders of magnitude smaller than frontier LLM weight-dumps. In principle we could work our way through these systems eventually, sure, maybe even a single person could do so. But if we really understand exactly how our software works, how have we been…
Oh that’s ez, for software we know that has bugs, one reason could be the cost of fixing isn’t worth it.
Re: “Next-token predictor” is the wrong mental model for LLMs
#239Earlier quoted context omitted.
> Nobody says a system or product works a certain way and means the system while it's half built. "Bridges drop cars in the water!". Right. To understand how an engine works, it's important to understand what a piston does as part of the engine.
You are conflating "half built" with "a piece of a system". The model weights change as the model goes through the training process. They aren't stored after pre-training is done and other weights are put somewhere else. It's more like pottery - the thing changes. It's not correct to say something is soft and malleable because it once was.
You're using the fact the both parts of training affect the same weights to support your argument that they're making the system do something fundamentally different after RL?
Re: “Next-token predictor” is the wrong mental model for LLMs
#240Earlier quoted context omitted.
You don't need to build a car to understand one. That you tie yourself up in knots of fancy acronyms instead of plain words and that your argument boils down to semantics of the word prediction, it's pretty clear what is up brother.
Lol, sure, just read a blog post and you'll understand how a car works....It's very simple....
Just like how reading a math book doesn’t teach you math, why do they make us read anyway? (Sarcasm) if reading a blog post didn’t teach someone how a car works how come it “can” work for next token predictors