Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Next-token predictor” is the wrong mental model for LLMs

gmcgoldr.github.io

Post reply on HN
Print view View original

41–50 of 320 posts

Re: “Next-token predictor” is the wrong mental model for LLMs

#41

Post by otabdeveloper4 »

atleastoptimal wrote: post #15

"Next-token predictor" is one of those phrases used most of the time with a motive to downplay the abilities and faculties of AI models. It is intended to trivialize LLM's and imply that there is some fundamental limit on their capacities. Relying on it as a mental model for what LLM's are minimizes the emergent properties of scaling. It's like imagining that unicellular life could never eventually evolve into comple…

That's literally what LLMs are.

No amount of cope and anthropomorphizing is gonna change that cold, hard fact.

P.S. The perceived magic of LLMs comes from the way they cross-correlate all the probabilities of tokens on their context window. Not from their ability to "think ahead". They can't do that by design.

Re: “Next-token predictor” is the wrong mental model for LLMs

#42

Post by otabdeveloper4 »

pjerem wrote: post #25
atleastoptimal wrote: post #15

"Next-token predictor" is one of those phrases used most of the time with a motive to downplay the abilities and faculties of AI models. It is intended to trivialize LLM's and imply that there is some fundamental limit on their capacities. Relying on it as a mental model for what LLM's are minimizes the emergent properties of scaling. It's like imagining that unicellular life could never eventually evolve into comple…

Good example. It’s also like saying our brains are just electric circuitry incorporated in meat. It’s true but it seems that consciousness emerges from this. The fact that LLMs are next token predictors isn’t the interesting or impressive part. Actually my brain strictly is a black box predicting (or choosing) my next word/action/move… based on a complex existing context (my thoughts, the environment, my physical sta…

> It’s true

It's not. "Brains as electrical circuits" is a gross simplification based on our ignorance and prejudices. (In the 18th century they spoke of brains as "clockwork mechanisms".)

LLMs, in contrast, are literally next token predictors. We know exactly how LLMs work, and they are exactly that.

Re: “Next-token predictor” is the wrong mental model for LLMs

#43

Post by danielmarkbruce »

The biggest problem is the word "predictor". Once you get into post training with RLHF and RLVR, it simply isn't doing that. It is not predicting anything. It's producing tokens, but it isn't predicting them. The chess analogy in the post is a good one - it's closer to searching for a set of moves that give a result than predict. It's search for a set of ideas, represented as locations in very high dimensional space, that when put together in the right order lead to a result.

Re: “Next-token predictor” is the wrong mental model for LLMs

#44

Post by otabdeveloper4 »

mmoll wrote: post #32
pjerem wrote: post #25

Earlier quoted context omitted.

Good example. It’s also like saying our brains are just electric circuitry incorporated in meat. It’s true but it seems that consciousness emerges from this. The fact that LLMs are next token predictors isn’t the interesting or impressive part. Actually my brain strictly is a black box predicting (or choosing) my next word/action/move… based on a complex existing context (my thoughts, the environment, my physical sta…

That is the point: our minds are also next-„token“-predictors, at least we can‘t prove they‘re not. That‘s why I don‘t agree with the article: LLMs _are_ next-token predictors. However, that says little about their capabilities. Also, while I have no idea what „consciousness“ is, I have difficulties believing that it could arise in a program that, in theory, you could execute with pen and paper.

We also can't prove that our minds aren't machine elf meat puppets. Come on. Please.

Re: “Next-token predictor” is the wrong mental model for LLMs

#45

Post by junofan »

mort96 wrote: post #20
atleastoptimal wrote: post #15

"Next-token predictor" is one of those phrases used most of the time with a motive to downplay the abilities and faculties of AI models. It is intended to trivialize LLM's and imply that there is some fundamental limit on their capacities. Relying on it as a mental model for what LLM's are minimizes the emergent properties of scaling. It's like imagining that unicellular life could never eventually evolve into comple…

At the same time, it ... is literally a next token predictor. Like that's what it is. The input is a sequence of tokens. The output is a probability distribution of next tokens.

Vacuous, like calling a V8 a “next piston firing predictor” because engines are designed so that one piston sets up the next in the firing order and technically there’s some nonzero probability any piston can (mis)fire next. It’s missing two pieces:

1. Useful work that has been done (the previously generated token sequence :: the mechanical work already accomplished)

2. The role of structure in relation to the application (post-training :: other components like crankshaft etc)

Re: “Next-token predictor” is the wrong mental model for LLMs

#46

Post by leni536 »

weego wrote: post #23
atleastoptimal wrote: post #15

"Next-token predictor" is one of those phrases used most of the time with a motive to downplay the abilities and faculties of AI models. It is intended to trivialize LLM's and imply that there is some fundamental limit on their capacities. Relying on it as a mental model for what LLM's are minimizes the emergent properties of scaling. It's like imagining that unicellular life could never eventually evolve into comple…

imply that there is some fundamental limit on their capacities This is a wildly dismissive statement that does a lot of heavy lifting. Your assertion is that we just happened to hit on a methodology that has no limitations between being an encyclopedia with a novel human language interface and, I guess by implication, AGI? That seems more outrageous a claim than the one you're dismissing.

Next token prediction is just an interface. It can be backed by a Markov chain, a neural model or an actual human being.

Re: “Next-token predictor” is the wrong mental model for LLMs

#47

Post by danielmarkbruce »

jameshart wrote: post #40
saghm wrote: post #28

Earlier quoted context omitted.

Yeah, it sounds like this is just a disagreement about what the word "next" means. I agree with you that "next" just means "the one about to come", and if the underlying model works by using some prediction mechanism to determine that, then it's by definition a next-token predictor. Disagreeing with that on the basis that the "next" token isn't necessarily in the training data verbatim just seems like an overly stric…

I think it’s a disagreement about what ‘predict’ means. The OP is arguing against people who think that an LLM is ‘predicting’ what token would likely follow if the text preceding were found among the corpus it was originally trained on . Instead it is ‘predicting’ what token would follow if the text were found among really good examples of the text it has being reinforced to produce - be that ‘chats with a helpful a…

yes, it's exactly this. And it's not a trivial distinction.

Re: “Next-token predictor” is the wrong mental model for LLMs

#48

Post by saghm »

jameshart wrote: post #40
saghm wrote: post #28

Earlier quoted context omitted.

Yeah, it sounds like this is just a disagreement about what the word "next" means. I agree with you that "next" just means "the one about to come", and if the underlying model works by using some prediction mechanism to determine that, then it's by definition a next-token predictor. Disagreeing with that on the basis that the "next" token isn't necessarily in the training data verbatim just seems like an overly stric…

I think it’s a disagreement about what ‘predict’ means. The OP is arguing against people who think that an LLM is ‘predicting’ what token would likely follow if the text preceding were found among the corpus it was originally trained on . Instead it is ‘predicting’ what token would follow if the text were found among really good examples of the text it has being reinforced to produce - be that ‘chats with a helpful a…

That seems overly pedantic to me. If I asked you "What's your prediction for the Super Bowl?", I'm pretty confident you would infer that I mean predicting the outcome, not the event itself.

Re: “Next-token predictor” is the wrong mental model for LLMs

#49

Post by uludag »

atleastoptimal wrote: post #15

"Next-token predictor" is one of those phrases used most of the time with a motive to downplay the abilities and faculties of AI models. It is intended to trivialize LLM's and imply that there is some fundamental limit on their capacities. Relying on it as a mental model for what LLM's are minimizes the emergent properties of scaling. It's like imagining that unicellular life could never eventually evolve into comple…

And what's wrong with downplaying the abilities and faculties of AI models if that's what people feel like saying? We don't call humans or animals sacks of chemicals because we believe they have moral status.

Re: “Next-token predictor” is the wrong mental model for LLMs

#50

Post by Aurornis »

mort96 wrote: post #20
atleastoptimal wrote: post #15

"Next-token predictor" is one of those phrases used most of the time with a motive to downplay the abilities and faculties of AI models. It is intended to trivialize LLM's and imply that there is some fundamental limit on their capacities. Relying on it as a mental model for what LLM's are minimizes the emergent properties of scaling. It's like imagining that unicellular life could never eventually evolve into comple…

At the same time, it ... is literally a next token predictor. Like that's what it is. The input is a sequence of tokens. The output is a probability distribution of next tokens.

This comment attracted a lot of analogies trying to reduce something to something else (calling humans a "bag of chemicals"), but the flaw in those analogies is that they're reducing something valuable to something that sounds less valuable.

With an LLM, the tokens are the valuable part. That's what I want from it. That's why it exists. The tokens are the point, and it produces those tokens one by one for me.

Post reply on HN