Live data from Hacker News

“Next-token predictor” is the wrong mental model for LLMs

gmcgoldr.github.io

71–80 of 307 posts

Re: “Next-token predictor” is the wrong mental model for LLMs

#71
post #20

Earlier quoted context omitted.

At the same time, it ... is literally a next token predictor. Like that's what it is. The input is a sequence of tokens. The output is a probability distribution of next tokens.

Yes, and by the same token, multicellular organisms are literally just sophisticated mitosis and survival optimizers for our cells. But when you take that optimization "to the limit" the cells develop weird things like body plans and back pain and Mozart. Both examples involve the same "aha" moment: even though it's true that you are literally 'just' doing XYZ, unbelievably complex patterns and sub-goals can emerge.

> and by the same token

I don't think you intended this, but the word choice here gave me a chortle.

Re: “Next-token predictor” is the wrong mental model for LLMs

#72

> Calling the second system a “next-move predictor” would be strange. It is not trying to predict what move appeared next in a dataset. It is trying to choose a move that wins. i dont understand the distinction here. does working backwards from a set of win states instead of working forwards from the current state somehow change whether it’s a prediction or not?

The word "predict" has a meaning. I don't "predict" my next move in chess. I might predict what someone elses first move is.

In any case this is all very pedantic. In the process of selecting a move to make there is a prediction. Whether that prediction is the opponent’s next move or what your next move should be based on the game’s existing state, there is a prediction that the next move you make will improve your chance to win. Maybe the probability in that selection is 100%. You have no other possible move. It doesn’t matter. All we are doing here as far as I can tell is arguing over where the prediction happens and whether that counts as predicting something.

Re: “Next-token predictor” is the wrong mental model for LLMs

#73
Calling an LLM a "next-token predictor" is like calling a TomTom a "next-turn predictor." It confuses the serial format of its instructions with the computation producing them, while ignoring the map, the route, the destination, and the goal -- as well as the people, businesses, traffic, and points of interest that make the map a model of an inhabited, changing world.

Re: “Next-token predictor” is the wrong mental model for LLMs

#74
post #28

Earlier quoted context omitted.

Yeah, it sounds like this is just a disagreement about what the word "next" means. I agree with you that "next" just means "the one about to come", and if the underlying model works by using some prediction mechanism to determine that, then it's by definition a next-token predictor. Disagreeing with that on the basis that the "next" token isn't necessarily in the training data verbatim just seems like an overly stric…

I think it’s a disagreement about what ‘predict’ means. The OP is arguing against people who think that an LLM is ‘predicting’ what token would likely follow if the text preceding were found among the corpus it was originally trained on . Instead it is ‘predicting’ what token would follow if the text were found among really good examples of the text it has being reinforced to produce - be that ‘chats with a helpful a…

Well said!

Re: “Next-token predictor” is the wrong mental model for LLMs

#75
post #59

Earlier quoted context omitted.

> But for many purposes you will not find it helpful to think of human beings as bags of chemicals But when we talk about humans, we're not talking about the chemicals involved in those humans. When we talk about LLMs, the tokens are the valuable thing they produce for us. We want LLMs because they give us sequences of tokens.

Agentic behaviors don't require end-users to be aware of tokens at all. Also, we literally say human actors have great chemistry :)

?? we pay for tokens though…

Re: “Next-token predictor” is the wrong mental model for LLMs

#76

To be honest, I believe I get the point the article is trying to make, and to an extent I agree, but I also think the point is not really made very well. The core of the argument as I understood it is that LLMs aren't just using existing data is training but also new ones. That's fine and good, and you can't simply assume an LLM is simply mashing together all it's data to give you an average of all that got fed into…

Maybe I'm wrong - but I see LLMs are a "next-token predictor" as somewhat equivalent to brains are a "bag of molecules".

Both systems have emergent behavior that goes well beyond what would naively be expected.

Re: “Next-token predictor” is the wrong mental model for LLMs

#77

To be honest, I believe I get the point the article is trying to make, and to an extent I agree, but I also think the point is not really made very well. The core of the argument as I understood it is that LLMs aren't just using existing data is training but also new ones. That's fine and good, and you can't simply assume an LLM is simply mashing together all it's data to give you an average of all that got fed into…

Blog articles from Anthropic and others show that this is not true.

A LLM already knows more tokens than the current one. It was mentioned in a blog post about how a LLM is doing haikus and co.

There are also structures in an LLM which allows it to 'estimate' numbers to a certain degree and doing other things.

Re: “Next-token predictor” is the wrong mental model for LLMs

#78
post #70
post #29

Earlier quoted context omitted.

It is. And human beings are bags of chemicals. But for many purposes you will not find it helpful to think of human beings as bags of chemicals, and for many purposes you will not find it helpful to think of LLMs as next-token predictors.

It can be pretty helpful to think of human function in chemical terms. Its at least unhelpful to deny it.

Milo Yiannopoulos used to think of other human beings as bags of chemicals until they deported his sack of shit molecules to the UK.

Re: “Next-token predictor” is the wrong mental model for LLMs

#79
post #29
post #20

Earlier quoted context omitted.

At the same time, it ... is literally a next token predictor. Like that's what it is. The input is a sequence of tokens. The output is a probability distribution of next tokens.

It is. And human beings are bags of chemicals. But for many purposes you will not find it helpful to think of human beings as bags of chemicals, and for many purposes you will not find it helpful to think of LLMs as next-token predictors.

Humans are next state of their local world predictors, given all previous states they are aware of. That's an entirely fair analogy. The reverse analogy for calling a human a bag of chemicals would be calling an LLM a sequence of bytes loaded from disk to memory, the most reductive possible description of any piece of software at all.

To be clear, all life is a next state of the local world predictor. What makes humans somewhat unique among life is we're much better at predicting states of the world neither we nor any of our ancestors have ever experienced, for various reasons such as having the ability to legibly communicate very complicated information strings to each other, being able to build and use tools to record states of the world we can't directly sense.

Similarly, what makes LLMs and multimodal versions of the same architectures "better" than previous generations of electronic predictive models is factors like being able to read and understand roughly the same corpus of data humans have been recording all these millennia, being able to read and remember much more of it than any individual human, and being better at generalizing than other electronic predictive models, but not better than humans. And, of course, they can produce far more predictions in far less time. Frankly, that is probably the key advantage that makes the Hacker News crowd love them so much. They're not any better at predicting byte strings that can be compiled or interpreted into executable code than humans are if you gave both infinite time to do it, but they're a lot faster.

Re: “Next-token predictor” is the wrong mental model for LLMs

#80

Earlier quoted context omitted.

The word "predict" has a meaning. I don't "predict" my next move in chess. I might predict what someone elses first move is.

In any case this is all very pedantic. In the process of selecting a move to make there is a prediction. Whether that prediction is the opponent’s next move or what your next move should be based on the game’s existing state, there is a prediction that the next move you make will improve your chance to win. Maybe the probability in that selection is 100%. You have no other possible move. It doesn’t matter. All we are…

There is no truth for RLHF or RLVR. You can't predict against something if you can't check against the truth.

It's not pedantry. The objective function changes. The optimization changes. THese are real things when training a model, not hand wavy philosophical ideas.

Post reply on HN