Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Next-token predictor” is the wrong mental model for LLMs

gmcgoldr.github.io

Post reply on HN
Print view View original

31–40 of 306 posts

Re: “Next-token predictor” is the wrong mental model for LLMs

#31

Post by atleastoptimal »

jvanderbot wrote: post #19
atleastoptimal wrote: post #15

"Next-token predictor" is one of those phrases used most of the time with a motive to downplay the abilities and faculties of AI models. It is intended to trivialize LLM's and imply that there is some fundamental limit on their capacities. Relying on it as a mental model for what LLM's are minimizes the emergent properties of scaling. It's like imagining that unicellular life could never eventually evolve into comple…

But it is a next token predictor. Recursively invoked. With carefully selected context. And massive investment in RL to tune token selection. And the ability to use cli tools on other folks' machines. That's a powerful system built around a conceptually simple technology: Next token predictors.

Yes this is correct. The thing is not about the term next-token predictor being correct, but because of the connotative weight of that phrase as a implicit trivialization of LLM abilities, which is how it is often used.

Re: “Next-token predictor” is the wrong mental model for LLMs

#32

Post by mmoll »

pjerem wrote: post #25
atleastoptimal wrote: post #15

"Next-token predictor" is one of those phrases used most of the time with a motive to downplay the abilities and faculties of AI models. It is intended to trivialize LLM's and imply that there is some fundamental limit on their capacities. Relying on it as a mental model for what LLM's are minimizes the emergent properties of scaling. It's like imagining that unicellular life could never eventually evolve into comple…

Good example. It’s also like saying our brains are just electric circuitry incorporated in meat. It’s true but it seems that consciousness emerges from this. The fact that LLMs are next token predictors isn’t the interesting or impressive part. Actually my brain strictly is a black box predicting (or choosing) my next word/action/move… based on a complex existing context (my thoughts, the environment, my physical sta…

That is the point: our minds are also next-„token“-predictors, at least we can‘t prove they‘re not. That‘s why I don‘t agree with the article: LLMs _are_ next-token predictors. However, that says little about their capabilities. Also, while I have no idea what „consciousness“ is, I have difficulties believing that it could arise in a program that, in theory, you could execute with pen and paper.

Re: “Next-token predictor” is the wrong mental model for LLMs

#33

Post by hackinthebochs »

weego wrote: post #23
atleastoptimal wrote: post #15

"Next-token predictor" is one of those phrases used most of the time with a motive to downplay the abilities and faculties of AI models. It is intended to trivialize LLM's and imply that there is some fundamental limit on their capacities. Relying on it as a mental model for what LLM's are minimizes the emergent properties of scaling. It's like imagining that unicellular life could never eventually evolve into comple…

imply that there is some fundamental limit on their capacities This is a wildly dismissive statement that does a lot of heavy lifting. Your assertion is that we just happened to hit on a methodology that has no limitations between being an encyclopedia with a novel human language interface and, I guess by implication, AGI? That seems more outrageous a claim than the one you're dismissing.

The stacked transformer paradigm picks out points in circuit design space. It is very possible this architecture has no inherent limitations on what it can compute in principle.

Re: “Next-token predictor” is the wrong mental model for LLMs

#34

Post by astro1234 »

weego wrote: post #23
atleastoptimal wrote: post #15

"Next-token predictor" is one of those phrases used most of the time with a motive to downplay the abilities and faculties of AI models. It is intended to trivialize LLM's and imply that there is some fundamental limit on their capacities. Relying on it as a mental model for what LLM's are minimizes the emergent properties of scaling. It's like imagining that unicellular life could never eventually evolve into comple…

imply that there is some fundamental limit on their capacities This is a wildly dismissive statement that does a lot of heavy lifting. Your assertion is that we just happened to hit on a methodology that has no limitations between being an encyclopedia with a novel human language interface and, I guess by implication, AGI? That seems more outrageous a claim than the one you're dismissing.

Well I think in the absence of convincing pieces of evidence to the contrary you might be right. You’re making an empirical statement but we have already answered it today:

- we get novel, emergent properties and capabilities of these models that were not trained

- they have very clear generalization to out of domain problems

The point is people conflate the end product: a model that can clearly do very novel, useful and interesting things, with the vehicle for getting there which is a series of optimization steps involving next token prediction loss.

You mention limitations; we all clearly know the practical limitations of these models today, but if you look at scaling laws and empirical performance trends (epoch capability index for example) as well as the trajectory over the last couple of years (very stable), the claim that there is some sort of fundamental limitation is now surprisingly the claim that has the burden of proof.

You can claim it may be e.g. finite context. That is fundamentally bad for certain classes of tasks. This was the hypothesis of a lot of lab leadership of urgently trying to anticipate how to get around this bottleneck (still of course lots of work on this) but the surprising thing is it does not appear to be at this point a blocker.

Re: “Next-token predictor” is the wrong mental model for LLMs

#37

Post by major__4 »

aeve890 wrote: post #14
Sprotch wrote: post #6

I understand how a computer can know that a chess move is more likely to lead to a win, and therefore “correct”, but I don’t understand how it can know that a token is correct. Can someone explain?

>I don’t understand how it can know that a token is correct. It can't. The next token is just the most statistically probably given the context (at least in transformers). Try a very small/weak model in your own machine and more often than not it would get stuck repeating the same word or even just output garbage. Because after training and quantization (where some information is lost), that's the most probable next…

classical particles obey deterministic state evolution rules, yet put 10 in a box and you cannot say where they will be 5 minutes later.

Re: “Next-token predictor” is the wrong mental model for LLMs

#38

Post by schiffern »

mort96 wrote: post #20
atleastoptimal wrote: post #15

"Next-token predictor" is one of those phrases used most of the time with a motive to downplay the abilities and faculties of AI models. It is intended to trivialize LLM's and imply that there is some fundamental limit on their capacities. Relying on it as a mental model for what LLM's are minimizes the emergent properties of scaling. It's like imagining that unicellular life could never eventually evolve into comple…

At the same time, it ... is literally a next token predictor. Like that's what it is. The input is a sequence of tokens. The output is a probability distribution of next tokens.

Yes, and by the same token, multicellular organisms are literally just sophisticated mitosis and survival optimizers for our cells. But when you take that optimization "to the limit" the cells develop weird things like body plans and back pain and Mozart.

Both examples involve the same "aha" moment: even though it's true that you are literally 'just' doing XYZ, unbelievably complex patterns and sub-goals can emerge.

Re: “Next-token predictor” is the wrong mental model for LLMs

#39

Post by chrisjj »

hirvi74 wrote: post #9

Sure, I get the gist of the article. I have never liked the reductionist argument that LLMs are nothing more than next-token predictors. By that rational, the human brain is really not that much different. When I am having a conversation with another person, I do not usually have every word I will respond with stored in my limited working memory. My output is often predicted based on the previous word I spoke.

> I have never liked the reductionist argument that LLMs are nothing more than next-token predictors.

I have never heard such an argument. Recognition that LLMs are nothing more than next-token predictors does not come from reductionism. It comes from simply knowing how they work e.g. from viewing the inference code.

Re: “Next-token predictor” is the wrong mental model for LLMs

#40

Post by jameshart »

saghm wrote: post #28
Systemerror7A69 wrote: post #21

To be honest, I believe I get the point the article is trying to make, and to an extent I agree, but I also think the point is not really made very well. The core of the argument as I understood it is that LLMs aren't just using existing data is training but also new ones. That's fine and good, and you can't simply assume an LLM is simply mashing together all it's data to give you an average of all that got fed into…

Yeah, it sounds like this is just a disagreement about what the word "next" means. I agree with you that "next" just means "the one about to come", and if the underlying model works by using some prediction mechanism to determine that, then it's by definition a next-token predictor. Disagreeing with that on the basis that the "next" token isn't necessarily in the training data verbatim just seems like an overly stric…

I think it’s a disagreement about what ‘predict’ means.

The OP is arguing against people who think that an LLM is ‘predicting’ what token would likely follow if the text preceding were found among the corpus it was originally trained on.

Instead it is ‘predicting’ what token would follow if the text were found among really good examples of the text it has being reinforced to produce - be that ‘chats with a helpful assistant’ or ‘sets of changes to a codebase’.

And that isn’t really ‘prediction’, so much as ‘generation’.

It’s not been tuned to ‘guess the next token right’. It’s been tuned to generate the token that leads to it ultimately scoring highest on its reward function.

It’s not predicting the token, it’s predicting the reward.

Post reply on HN