Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Next-token predictor” is the wrong mental model for LLMs

gmcgoldr.github.io

Post reply on HN
Print view View original

271–280 of 330 posts

Re: “Next-token predictor” is the wrong mental model for LLMs

#271

Post by [deleted] »

hirvi74 wrote: post #24
Sprotch wrote: post #6

I understand how a computer can know that a chess move is more likely to lead to a win, and therefore “correct”, but I don’t understand how it can know that a token is correct. Can someone explain?

My poor understanding is that an LLM does not "know" either. It basically uses probabilities to predict the next word based on a large matrix of probable outcomes. For example, say I ask an LLM, "What sentence in English contains every letter in the alphabet?" It would respond with something like: "The quick fox jumps over the lazy, brown [next word]" (Assume all the words were previously guessed correctly at this po…

[deleted]

Re: “Next-token predictor” is the wrong mental model for LLMs

#272

Post by bjourne »

saghm wrote: post #28
Systemerror7A69 wrote: post #21

To be honest, I believe I get the point the article is trying to make, and to an extent I agree, but I also think the point is not really made very well. The core of the argument as I understood it is that LLMs aren't just using existing data is training but also new ones. That's fine and good, and you can't simply assume an LLM is simply mashing together all it's data to give you an average of all that got fed into…

Yeah, it sounds like this is just a disagreement about what the word "next" means. I agree with you that "next" just means "the one about to come", and if the underlying model works by using some prediction mechanism to determine that, then it's by definition a next-token predictor. Disagreeing with that on the basis that the "next" token isn't necessarily in the training data verbatim just seems like an overly stric…

The argument is that a modern LLM cares little for the MLE. Hence, statistically speaking, they are not predictors.

Re: “Next-token predictor” is the wrong mental model for LLMs

#273

Post by [deleted] »

WarmWash wrote: post #109
Kranar wrote: post #106

Earlier quoted context omitted.

You're misinterpreting these articles. Autoregressive LLMs generate tokens one at a time, disputing this is just plain wrong. What is true, however, is that in order to generate the next token autoregressive LLMs produce internal/hidden state about future tokens far past the next token so that it's not like the entire machinery of the algorithm deprives itself of representing where the sentence/text is headed. So "em…

I think the snag is that people use "Next token predictor" as a dismissive statement. In reality it's about as functionally dismissive as "humans only communicate one word at a time".

[deleted]

Re: “Next-token predictor” is the wrong mental model for LLMs

#274

Post by ruszki »

Systemerror7A69 wrote: post #21

To be honest, I believe I get the point the article is trying to make, and to an extent I agree, but I also think the point is not really made very well. The core of the argument as I understood it is that LLMs aren't just using existing data is training but also new ones. That's fine and good, and you can't simply assume an LLM is simply mashing together all it's data to give you an average of all that got fed into…

But before ChatGPT,, models had already done that, like all the time, and nobody questioned that these types of models (LLMs) wouldn't be next token predictors, since this is just an inference input data topic. This is questioned only since some less knowledgeable people seem to not have the vocabulary to express emergent properties of large next token predictors.

Reading the article, they wanted to convey that the loss function is more complicated... But they are still next token predictors, just not the trivial ones. Unfortunately, that was true even before, because the input data had to be cleared even decades ago, so there is nothing new. This article just butt hurt that some people deny that there are emergent properties with those, and try to sell something trivial in the field for many decades now. Current LLMs are not different because of these. Also, if they try to sell generated data in the training set, then that's also not new at all.

Re: “Next-token predictor” is the wrong mental model for LLMs

#275

Post by [deleted] »

danielmarkbruce wrote: post #250
doc_ick wrote: post #244

Earlier quoted context omitted.

Just skimming through here but I think you have the wrong ideas with llms, I’d recommend Andrew Ngs course (correct me if you’ve already seen it or something similar).

So, this is the cause of the problem.... People take an intro to LLMs course, follow happily along, and don't realize there is more to it than the next token prediction. And those courses teach how LLMs were built in 2017-2020 maybe. Then RL got added to the mix. The current models really are very different to the models from then - everything that is now considered "post-training" isn't doing next token prediction.

[deleted]

Re: “Next-token predictor” is the wrong mental model for LLMs

#276

Post by gwerbin »

danielmarkbruce wrote: post #214
gwerbin wrote: post #209

Earlier quoted context omitted.

They're estimating a probability distribution over the next token, from which a sample is taken. Close enough.

It's not an estimation of something. It's a policy.

Sure, you're right.

But it's a policy learned from a next-token prediction task. You could also call it an inferrer or generator or whatever. The point is that it takes as input a sequence of preceding tokens and emits one more token to continue the sequence.

Re: “Next-token predictor” is the wrong mental model for LLMs

#277

Post by garrinm »

Alpha3031 wrote: post #208
garrinm wrote: post #170

Earlier quoted context omitted.

I think the point is more that in RL there's no ground truth to predict. So when training a model with RL the idea of "predicting" doesn't fit anymore. I'll make some edits I see that I wasn't very clear.

I feel like RLHF has a pretty obvious ground truth, human feedback is used as an (albeit noisy) signal of average human preferences. Same thing with RLVR and "solving the problem".

To be more specific there’s no ground truth tokens to predict. There a verifiable answer in RLVR. But the tokens are explored. Not predicted as there’s no true token to predict.

Re: “Next-token predictor” is the wrong mental model for LLMs

#279

Post by chrisjj »

ozgung wrote: post #265
chrisjj wrote: post #260

Earlier quoted context omitted.

> you can’t understand how can they create a civilization They don't.

I used 'civilization' as the popular term for agent swarms in the OpenAI incident. https://www.dwarkesh.com/p/openai-huggingface https://calnewport.com/are-we-at-war-with-ai-agent-civilizat... Isolated agents invented a way to communicate between each other and started to collaborate, organize and conspire by themselves. No human prompted them to do that or taught them how to do that. BTW, Autocomplete just keeps the…

> I used 'civilization' as the popular term for agent swarms in the OpenAI incident.

s/popular/unpopular/. Even OpenAI's PR dept. called it only a collective.

> Isolated agents invented a way to communicate between each other

So, not isolated.

> and started to collaborate, organize and conspire by themselves.

So, no civilisation.

Re: “Next-token predictor” is the wrong mental model for LLMs

#280

Post by epistasis »

chrisjj wrote: post #266
epistasis wrote: post #247

Earlier quoted context omitted.

I wouldn't necessarily say that. Anybody who's playing a chess game is predicting their next move, whether or not they're inventing new moves. LLMs are not simple things like a Markov model, there's a lot going on in there, it's not deterministic, and it's completely capable of creating entire new styles of play based on complex interactions of internal states.

> LLMs are not simple things like a Markov model "Large Language Models as Markov Chains" https://www.alphaxiv.org/pdf/2410.02724

Argh, what I wrote was obviously wrong. What I meant was to refer to the simple n-state Markov models as used past decades.

Anything that's sequential, like language is, will exhibit Markovian properties, and be somewhat a "Markov" model.

Markov chains are a different concept than a Markov model, but I do agree that, technically, an LLM is a Markov model, just with an internal state space that is nothing like what is usually meant when ML people refer to Markov models.

Post reply on HN