Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Next-token predictor” is the wrong mental model for LLMs

gmcgoldr.github.io

Post reply on HN
Print view View original

281–290 of 330 posts

Re: “Next-token predictor” is the wrong mental model for LLMs

#281

Post by chrisjj »

HarHarVeryFunny wrote: post #108
chrisjj wrote: post #10

Better title: Continue thinking of LLMs as Next-Token Predictors Because no, post training doesn't change that.

I'm not sure that's a useful way to think of it. RL post-training changes the nature of what is being predicted, basically turning it from a copying machine into a goal-seeking machine. A base model is predicting training sample continuations (copying). A post-trained model is now steering/narrowing the base model's predictions in directions that were reinforced by RL goals. The model is no longer predicting what the…

> The model is no longer predicting what the next token will be, but rather predicting what it should be in order to steer generation in the reinforced directions.

So still next-token prediction, then.

Re: “Next-token predictor” is the wrong mental model for LLMs

#282

Post by perrygeo »

"LLMs are next-token predictors" is a perfectly accurate mental model. But that doesn't preclude higher-level models such as "LLMs emulate artificial general intelligence". Both can be true.

In systems, we can have facts which emerge from other facts at different levels of abstraction. The causal relationship is not linear. It's not entirely clear that next-token prediction should result in anything close to "intelligence". Yet it does.

Life is another good example. Some might say "biology is just organic chemistry" while others might say "biology is an interconnected planetary system which captures low entropy energy". Both are true.

As a result of emergent phenomenon, we have to take the stance of explanatory pluralism; using the explanation that works best in context. There is no single mental model that works everywhere.

I will continue to think of LLMs as next-token predictors because it's (sometimes) useful, and empirically true. But I also think of them as "pattern matchers", searching for language patterns and trying to replicate them. This is also (sometimes) useful and empirically true. There's likely an infinite number of mental models; our job is to pick one that's both true and useful.

Re: “Next-token predictor” is the wrong mental model for LLMs

#283

Post by NotHowItWorks »

It literally is token prediction with vector search.

Yes there’s an app layer in the chat product for convenience and parsing but the model is exposing methods like ‘complete’ that predict out word sequences.

When model temperature settings are not added in, you get the same exact response every time, just like ngram.

What you’re seeing with Gemini and ChatGPT is context caching to prevent getting sidetracked and response boilerplates with multi-modality so they can call out to image generation, a code completion service, etc. to assemble a complete response.

That’s why the mainstream providers seem so much better (or at least consistent in replies) - each prompt is actually requesting multiple models and doing a lot of application level work to assemble the response you see. That also why they take sooo long to respond.

The language model itself is quite literally a text completion machine, with settings.

Re: “Next-token predictor” is the wrong mental model for LLMs

#284

Post by doc_ick »

danielmarkbruce wrote: post #250
doc_ick wrote: post #244

Earlier quoted context omitted.

Just skimming through here but I think you have the wrong ideas with llms, I’d recommend Andrew Ngs course (correct me if you’ve already seen it or something similar).

So, this is the cause of the problem.... People take an intro to LLMs course, follow happily along, and don't realize there is more to it than the next token prediction. And those courses teach how LLMs were built in 2017-2020 maybe. Then RL got added to the mix. The current models really are very different to the models from then - everything that is now considered "post-training" isn't doing next token prediction.

Please feel free to cite sources then, otherwise I see no relevancy from you.

Re: “Next-token predictor” is the wrong mental model for LLMs

#285

Post by simianwords »

huurtehoog wrote: post #261
highfrequency wrote: post #86

Earlier quoted context omitted.

Sure, politics and economics are involved - but why can’t technology also play an important role? None of this AI political economy stuff was relevant in 2015 because necessary technological breakthroughs had not yet been made.

Computers have been at the center of civilization and politics since they helped win a war by changing intelligence work forever in WWII. The idea that some new thing was born with LLMs and that this new thing fundamentally changes the calculus from the politics of labor and wealth into a technical discussion of possibilities and constraints determined by what the machine is and can do, and not what people should do,…

Not true, it is usually technology that changes politics and not the other way round (though it is reflexive in practice).

The type of politics possible under agriculture is different from one in industrial age.

Re: “Next-token predictor” is the wrong mental model for LLMs

#286

Post by simianwords »

aeve890 wrote: post #14
Sprotch wrote: post #6

I understand how a computer can know that a chess move is more likely to lead to a win, and therefore “correct”, but I don’t understand how it can know that a token is correct. Can someone explain?

>I don’t understand how it can know that a token is correct. It can't. The next token is just the most statistically probably given the context (at least in transformers). Try a very small/weak model in your own machine and more often than not it would get stuck repeating the same word or even just output garbage. Because after training and quantization (where some information is lost), that's the most probable next…

> It can't.

Neither can you. This is the whole debate.

Re: “Next-token predictor” is the wrong mental model for LLMs

#287

Post by [deleted] »

danielmarkbruce wrote: post #193
dijksterhuis wrote: post #187

Earlier quoted context omitted.

From GP, i.e. the context for this local part of the thread > Autoregressive LLMs generate tokens one at a time, disputing this is just plain wrong. next-token prediction i.e. the bit built during pre-training. at no point in your reply to GP did you specify that you were referring to post-training. respectfully, it seems like this one is on you pal :shrug: > GPT-2 didn't use any reinforcement learning and is often g…

All modern LLMs that actually get used go through post-training. The finished product is something which has been through post training. So they are not next token prediction machines.

[deleted]

Re: “Next-token predictor” is the wrong mental model for LLMs

#289

Post by HarHarVeryFunny »

chrisjj wrote: post #281
HarHarVeryFunny wrote: post #108

Earlier quoted context omitted.

I'm not sure that's a useful way to think of it. RL post-training changes the nature of what is being predicted, basically turning it from a copying machine into a goal-seeking machine. A base model is predicting training sample continuations (copying). A post-trained model is now steering/narrowing the base model's predictions in directions that were reinforced by RL goals. The model is no longer predicting what the…

> The model is no longer predicting what the next token will be, but rather predicting what it should be in order to steer generation in the reinforced directions. So still next-token prediction, then.

Not really - after RL the probability distribution output by the model is no longer a prediction driven by the statistics of the training data, but rather whatever was reinforced as part of a synthetic generation that lead to a reward during RL post-training. As Karpathy notes, these reinforced token-sequences (from current token out to some future reward many tokens in the future) may even include errors and backtracking, with even the errors being reinforced, so not only is this synthetic data but it may even be incorrect synthetic data being reinforced.

So, I think "steering" is the best description of it - these aren't real training sample continuations being predicted, but rather RL tweaking a whole path of generation probabilities out into the future, with the hope being that if sampling follows this (stochastic) direction then it may eventually lead to a "correct" response.

Thinking of it as goal seeking also provides some insight into the resulting behavior of the model.

Re: “Next-token predictor” is the wrong mental model for LLMs

#290

Post by chrisjj »

HarHarVeryFunny wrote: post #289
chrisjj wrote: post #281

Earlier quoted context omitted.

> The model is no longer predicting what the next token will be, but rather predicting what it should be in order to steer generation in the reinforced directions. So still next-token prediction, then.

Not really - after RL the probability distribution output by the model is no longer a prediction driven by the statistics of the training data, but rather whatever was reinforced as part of a synthetic generation that lead to a reward during RL post-training. As Karpathy notes, these reinforced token-sequences (from current token out to some future reward many tokens in the future) may even include errors and backtra…

> the probability distribution output by the model is no longer a prediction driven by the statistics of the training data, but rather whatever was reinforced as part of a synthetic generation that lead to a reward during RL post-training.

Who cares? Still it is next-token prediction.

Post reply on HN