Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“Next-token predictor” is the wrong mental model for LLMs

gmcgoldr.github.io

Post reply on HN
Print view View original

311–320 of 333 posts

Re: “Next-token predictor” is the wrong mental model for LLMs

#311

Post by chrisjj »

chrisjj wrote: post #301
HarHarVeryFunny wrote: post #292

Earlier quoted context omitted.

The question being originally asked is whether "next-token predictor" is the right mental model for an RL-trained model, and I think the answer is no - not only is it not technically correct, but it is a misleading mental model and will lead to incorrect expectations/explanations of what the model is doing. Calling the base model a next token predictor is accurate since it is literally making a prediction and being g…

> The question being originally asked is whether "next-token predicton" is the right mental model for an RL-trained model, Regardless, the statement being challenged here is "still next-token prediction, then". > and I think the answer is no - not only is it not technically correct It is correct. RL simply adjusts weights - with no effect beyond an equivalent adjustment to the corpus itself. Hence "next-token predict…

> why do you find "next token predictor" to be the most useful mental model?

To me it is an accurate description of the algorithm. And a sufficient explanation for the behaviour. So I don't need it or anything else as a mental model.

I accept this does not suffice for people who cannot comprehend the huge amount of processing and data the empowers it. Lacking a factual understanding, they reach for any mental model as a kind of superstition.

It is sufficiently advanced technology which to many is indistinguishable from magic. This disguises its limitations and enables its limitless false promotion to the gullible, being the reason it is so dangerous to individuals and society.

Re: “Next-token predictor” is the wrong mental model for LLMs

#312

Post by danielmarkbruce »

grebc wrote: post #308
danielmarkbruce wrote: post #296

Earlier quoted context omitted.

Mine was sarcasm. People who actually understand cars have built them. Until you build something, you don't understand it.

Now you’re claiming people don’t understand unless they build something. Boy, oh boy, do you keep digging your logic hole that much deeper. As mentioned earlier, Sam thanks you for your obfuscation efforts while his equity keeps going up. The swindle continues.

I'm not the one hiding behind a fake name.

If you want to understand how this stuff works, there are totally decent books about building them from scratch. It's not that hard, and you'll likely find it interesting. Sebastian Raschka and Nathan Lambert have good books out, and the Allen Institute has available all the code and data they have used for several projects.

Re: “Next-token predictor” is the wrong mental model for LLMs

#313

Post by grebc »

danielmarkbruce wrote: post #312
grebc wrote: post #308

Earlier quoted context omitted.

Now you’re claiming people don’t understand unless they build something. Boy, oh boy, do you keep digging your logic hole that much deeper. As mentioned earlier, Sam thanks you for your obfuscation efforts while his equity keeps going up. The swindle continues.

I'm not the one hiding behind a fake name. If you want to understand how this stuff works, there are totally decent books about building them from scratch. It's not that hard, and you'll likely find it interesting. Sebastian Raschka and Nathan Lambert have good books out, and the Allen Institute has available all the code and data they have used for several projects.

Now a fake name accusation is thrown by someone with three first names.

Keep digging that hole, I’m sure you’ll surface somewhere with some sunshine.

Re: “Next-token predictor” is the wrong mental model for LLMs

#314

Post by stevenhuang »

perrygeo wrote: post #282

"LLMs are next-token predictors" is a perfectly accurate mental model. But that doesn't preclude higher-level models such as "LLMs emulate artificial general intelligence". Both can be true. In systems, we can have facts which emerge from other facts at different levels of abstraction. The causal relationship is not linear. It's not entirely clear that next-token prediction should result in anything close to "intelli…

You might like this article about how large scale order emerges out of the small scale.

Definitely lends credence to the idea that however these models work, focusing so much on them being next token predictors may rather be incidental to deeper mechanisms behind their function.

> Some of these networks organize themselves into states that can reliably identify macroscopic patterns in data regardless of microscopic differences between the states of individual neurons in the network. The decision of which pattern will be output by the network “works at a higher level,” said Rosas.

https://www.quantamagazine.org/the-new-math-of-how-large-sca...

Re: “Next-token predictor” is the wrong mental model for LLMs

#316

Post by zahlman »

amluto wrote: post #158
zahlman wrote: post #115

Earlier quoted context omitted.

Even so, one might wonder why we don't try making systems that take different approaches. For example, after a traditional first pass of output, they could do sliding-window "optimizations" considering each token in the context of tokens both before and after , and possibly replace words or phrases in-place. For example, I've noticed quite a few cases recently of LLMs outputting "but" where "and" would make more sens…

Look up diffusion models.

Indeed; but I've only heard of them being used for images rather than text. Why?

Re: “Next-token predictor” is the wrong mental model for LLMs

#317

Post by YeGoblynQueenne »

danielmarkbruce wrote: post #303
YeGoblynQueenne wrote: post #300

Earlier quoted context omitted.

Eh, no, that's not right. I might need to brush up on my Sutton & Barto but the RL task is traditionally defined as, informally, "given a current state observation predict the next action, state and reward". A policy is always predicting the next timestep's reward. Otherwise, how would it know what to do next?

Brush up :) The policy is optimized to maximize the the total reward, defined as the sum of the reward at each step, discounted by some factor.

Alright, I'll have to check up on that. Thanks for being nice about it.

Re: “Next-token predictor” is the wrong mental model for LLMs

#318

Post by huurtehoog »

simianwords wrote: post #285
huurtehoog wrote: post #261

Earlier quoted context omitted.

Computers have been at the center of civilization and politics since they helped win a war by changing intelligence work forever in WWII. The idea that some new thing was born with LLMs and that this new thing fundamentally changes the calculus from the politics of labor and wealth into a technical discussion of possibilities and constraints determined by what the machine is and can do, and not what people should do,…

Not true, it is usually technology that changes politics and not the other way round (though it is reflexive in practice). The type of politics possible under agriculture is different from one in industrial age.

Technology has no agency. It's merely a word for all the knowledge and tools that people use. The root of it is what people do with and to each other. A bigger lever doesn't change the fundamental fact that there is a person using it to move other people. Whatever the tool, whatever the technique, the thing that people want to distract you from when they put technology front and center is that someone somewhere is making a choice that impacts other people when they use technology. It is always someone's choice that causes things to go how they go in society and in the economy, not the shape or nature of the tools or technique that they use.

Re: “Next-token predictor” is the wrong mental model for LLMs

#319

Post by ltbarcly3 »

ozgung wrote: post #252
ltbarcly3 wrote: post #181

Earlier quoted context omitted.

It's not simple to do next token prediction. That IS what is going on. You want something 'deep'. Deep things are often very not complicated. The deep realization is that if you can predict the next token well enough, you can do things like this: . And it turned out the killer was And if it's really good at predicting the next token, it has to understand the novel and the clues, which means understanding the context…

I understand generating tokens sequentially has many benefits. But not all AI models do next token prediction. World models, video/image models, even Diffusion Language Models don’t work like that. They do more like “all tokens at once prediction”. So “next token” is actually an engineering design choice. (Even the concept of “token” is a design choice. Inside the Transformer there are just activations/feature vector…

Reinforcement learning doesn't make them smarter though, it probably actually reduces their intelligence in some sense, but it does socialize them so that they will tend to behave in ways that are more useful.

For example, when you paste the first 30 lines of a famous speech, you don't want it to finish the speech, you want it to give you the identity and some analysis of what you just pasted. From what I understand, that is the reinforcement part.

Now that large AI vendors have a massive corpus of user interactions however, the lines have likely become more blurred.

Re: “Next-token predictor” is the wrong mental model for LLMs

#320

Post by garrinm »

simonw wrote: post #177

I really like the pseudo code example on this post - one of the clearest simplified explanations I've seen of how inference and training work.

Originally I had those parts written in math with probability functions and the likes (its closer to my background). Then I remembered who is my target audience... but now that I see exactly who is my target audience I'm thinking I'll should have snuck a pelican in there. All jokes aside I appreciate the comment and I'm glad that rewrite paid off!
Post reply on HN