Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Next-token predictor” is the wrong mental model for LLMs

gmcgoldr.github.io

Post reply on HN
Print view View original

141–150 of 330 posts

Re: “Next-token predictor” is the wrong mental model for LLMs

#141

Post by Planktonne »

WarmWash wrote: post #113
Planktonne wrote: post #105

Earlier quoted context omitted.

I don't that's a fair description of either 'next-token predictor' or 'stochastic parrot'. Both of those terms describe mechanism, not value--the fact that people squawk that the terms are minimising is projection on their part, not inherent to the phrase.

Words and expressions often have both colloquial and literal meanings. Trying to argue away the colloquial meaning under the guise of the literal meaning is just another form of whooshing.

Sure, but just because you think a colloquial meaning exists doesn't mean you're correct. It particularly doesn't mean that if you're seeing other people (that you don't like) use a term you yourself don't use, and you're jumping to conclusions about it without consideration.

Re: “Next-token predictor” is the wrong mental model for LLMs

#142

Post by zahlman »

Planktonne wrote: post #125
zahlman wrote: post #118

Earlier quoted context omitted.

> I'm not sure what you want me to do with that information For example, you could cite specific things that you believe to be "AI tells" or "admissions".

It's a short article; you could read it. One example to get you started is the very first sentence: > Strictly speaking, the statement “LLMs are next-token predictors” isn’t wrong, but it’s incomplete. The article is about how 'next-token predictor' is the wrong mental model; it opens with the admission that it is not the wrong mental model.

I did read it. People are allowed to disagree with your conclusions. Comment guidelines ask us all not to make such accusations.

To say that a statement is incomplete, but not strictly speaking wrong, is perfectly compatible with describing it informally as "wrong" in the sense used in the title (i.e.: "not the most appropriate possibility").

Re: “Next-token predictor” is the wrong mental model for LLMs

#143

Post by angoragoats »

danielmarkbruce wrote: post #116
ordersofmag wrote: post #104

Earlier quoted context omitted.

The LLM does not determine the next token. It generate odds for all of the tokens it knows as to their likelihood of being 'next'. It's up to the harness running the LLM (and in most cases the a temperature setting) to actually decide on a particular next token. I think it's more accurate to call the thing the LLM actually generates (an ensemble of probabilities) a 'prediction'. It might be accurate to say the harnes…

Respectfully, go build one, including doing RLHF and RLVR. Those phases generate lots of tokens, then get scored on the entirety of the output, then optimize based on a scoring of that output. It doesn't check a "prediction" against what was actually "next" in data, because there isn't any "next token" data it's training on.

> It doesn't check a "prediction" against what was actually "next" in data

Literally no one here is claiming that it does. This is one of the many flaws in the article.

Re: “Next-token predictor” is the wrong mental model for LLMs

#144

Post by zahlman »

garrinm wrote: post #133
Planktonne wrote: post #123

Earlier quoted context omitted.

In my experience, people who do 'AI-assisted' writing tend to be very bad at noticing how much of their work AI has changed. I'm sure you put thought into it, but passing it through AI takes a lot of that out.

I think that’s fair, I didn’t actually run the whole thing through an AI. it was more targeted edits, but each time it does erode at my writing. But at the same time, I don’t think it’s a good reason to dismiss this. Because I did spend several hours writing it, and I did put a lot of thought into it, and it was not in any meaningful way generated by AI.

Out of curiousity, do you ask your editing system for diffs? Seems to me like the best way to notice and review whether your "voice" is degrading.

Personally I would never let an LLM touch my prose (although I'd happily use it for research and paraphrase things it told me), but if I force myself to consider the idea, that seems like the first thing I'd want. Maybe upon reading a diff you'd even consider going a third way with the text.

Re: “Next-token predictor” is the wrong mental model for LLMs

#145

Post by jameshart »

saghm wrote: post #129
jameshart wrote: post #97

Earlier quoted context omitted.

But if you are an NFL coach and I ask you to decide your next action in order to maximize your odds of winning the superbowl , while yes that does involve you having some predictive ability to think about what impact your actions would have on your odds of winning the Super Bowl… I don’t think you would call the process that you use to decide that next action ‘prediction’.

I don't find the fact that I don't call any humans "action predictors" to be a particularly meaningful insight because my rationale is that it's a weird thing to call a human; football coaches can do plenty of other things besides just coaching football.

Yep, we’re all just putting one foot in front of the other, hoping we’re doing the right thing to bring about the outcomes we want, trying our best.

But that’s the point: so is an LLM. Putting one token in front of another, hoping it’s doing the right thing to bring about the rewards it’s trained to… trying its best.

So yeah, not ‘next token predictors’. ‘Next token tryers’ maybe.

Re: “Next-token predictor” is the wrong mental model for LLMs

#146

Post by Planktonne »

zahlman wrote: post #142
Planktonne wrote: post #125

Earlier quoted context omitted.

It's a short article; you could read it. One example to get you started is the very first sentence: > Strictly speaking, the statement “LLMs are next-token predictors” isn’t wrong, but it’s incomplete. The article is about how 'next-token predictor' is the wrong mental model; it opens with the admission that it is not the wrong mental model.

I did read it. People are allowed to disagree with your conclusions. Comment guidelines ask us all not to make such accusations. To say that a statement is incomplete, but not strictly speaking wrong, is perfectly compatible with describing it informally as "wrong" in the sense used in the title (i.e.: "not the most appropriate possibility").

To informally describe something as wrong in an article focused on how it's wrong to informally describe something is incoherent.

There's a certain irony in pointing me towards the guidelines on the grounds that I have limited patience with your comments that violate them in various ways. I'm not sure that this is a productive discussion.

Re: “Next-token predictor” is the wrong mental model for LLMs

#147

Post by wat10000 »

gwerbin wrote: post #137
Planktonne wrote: post #134

Earlier quoted context omitted.

> Describing it as a "next-token predictor" in the sense that this would mean it's fundamentally limited to just a fraction of an inferential step I don't think anyone is doing that though; we know LLMs are not simple Markov chains, and that the prediction they make is based on more than the previous X words. It's not minimising to describe even a complex prediction process as prediction.

But it literally is making a prediction based on the previous X tokens, it's just that X is huge and there is a proportionally huge number of parameters in the token generation function.

The discourse around this is annoying. A big group of people use "next-token predictor" to imply that LLMs aren't capable of anything interesting. Another big group of people opposes the use of "next-token predictor" because of that implication. But that fight isn't about the "predictor" language at all.

The linked article makes a good point: a substantial chunk of the training does not consist of "here's a bunch of tokens, here's the next token, learn that." But all the comments want to turn it into a referendum on the goodness of AI.

Re: “Next-token predictor” is the wrong mental model for LLMs

#149

Post by danielmarkbruce »

grebc wrote: post #130
danielmarkbruce wrote: post #112

Earlier quoted context omitted.

Nope, it doesn't. No logic required, you can just build an LLM yourself, including post training. You'll see that predicting the next token isn't something the model does or is optimized for in RLHF or RLVR. You can hand wave all you like, but you have never done it.

Yes, no logic is necessary for LLM adherents we're all finding out. Carry on good soldier.

If you haven't built one, and don't understand how they work, why comment?

Re: “Next-token predictor” is the wrong mental model for LLMs

#150

Post by garrinm »

angoragoats wrote: post #143
danielmarkbruce wrote: post #116

Earlier quoted context omitted.

Respectfully, go build one, including doing RLHF and RLVR. Those phases generate lots of tokens, then get scored on the entirety of the output, then optimize based on a scoring of that output. It doesn't check a "prediction" against what was actually "next" in data, because there isn't any "next token" data it's training on.

> It doesn't check a "prediction" against what was actually "next" in data Literally no one here is claiming that it does. This is one of the many flaws in the article.

It does in pre training, but not in RL post training. And not at inference time. Reading over all these comments I get the feeling my mistake was not clearly delineating inference time and train time.
Post reply on HN