Some thoughts on autoregressive models
wonderfall.dev
Some thoughts on autoregressive models
1–10 of 60 posts
Re: Some thoughts on autoregressive models
#2[dead]
Re: Some thoughts on autoregressive models
#3The author (and Chomsky) fail to understand that LLMs (as well as human brains) are not just autoregressive models, but nonlinear autoregressive models. Put a slightly different way, you can describe LLMs as autoregressive, but only by taking liberties with the classical definition of 'autoregressive.'
The human mind is not, like ChatGPT and its ilk, a lumbering statistical engine for pattern matching, gorging on hundreds of terabytes of data and extrapolating the most likely conversational response or most probable answer to a scientific question. On the contrary, the human mind is a surprisingly efficient and even elegant system that operates with small amounts of information; it seeks not to infer brute correlations among data points but to create explanations. – Noam Chomsky
It's as if Chomsky has either never heard of transformers, or doesn't understand what they do.
Before speaking a sentence, we have a general idea of what we’re going to say; we don’t really choose what to say next based on the last word. That kind of planning isn’t something that can be represented sequentially.
It's as if the author (and Chomsky) has never seen a CoT model in action.
Re: Some thoughts on autoregressive models
#4The author (and Chomsky) fail to understand that LLMs (as well as human brains) are not just autoregressive models, but nonlinear autoregressive models. Put a slightly different way, you can describe LLMs as autoregressive, but only by taking liberties with the classical definition of 'autoregressive.' The human mind is not, like ChatGPT and its ilk, a lumbering statistical engine for pattern matching, gorging on hun…
Author here and I welcome the feedback, but I don't really understand your point. My post is clearly not dismissive of efforts to make LLMs reason using CoT prompting techniques and post-training, and I think such efforts are even mentioned. The model remains autoregressive either way, and this reasoning is not some kind of magic that makes them behave differently - these improvements only make them perform (much) better on given tasks.
Additionally, I'm not dismissive of the non-linear nature of transformers which I'm familiar with. Attention mechanism is a lot more complex than a linear relationship between the prediction and the past inputs, yes. But the end result remains sequential prediction. Ironically, diffusion models are kind of the opposite: sequential internally, parallel prediction at each step.
(Note: added note on terminology since the confusion arised by my use of "linearity", which was not referring to the attention mechanism itself. I've read so many papers that are perfectly fine with the use of "autoregressive" for this paradigm that I forgot some people coming from traditional statistics may be confused. Also "based on the last word" was wrong and meant "last words" or "previous words", obviously.)
All that being said, I don't think it's fair to say one doesn't understand how transformers work solely because of semantic interpretation. I appreciate the feedback though!
Re: Some thoughts on autoregressive models
#5> The context window can be compared to working memory in humans: it’s fast, efficient but gets rapidly overloaded. Humans manage this limitation by offloading previously learned information into other memory forms, whereas LLMs can only mimic this process superficially at best.
This is just silly. Humans forget things all the time! If I want to remember something I write it down.
> The nature of hallucination is very different between AR models and humans, as one has a world model and the other doesn’t.
I stopped reading at this point. There's not much signal here, just basic facts about LLMs and then leaps to very bold statements.
Here is an interesting experiment I use to help people understand next token prediction. Think of a simple math problem in your head, maybe 3 digit by 2 digit multiplication. Then speak out every single thought you have while solving it.
Re: Some thoughts on autoregressive models
#6The author (and Chomsky) fail to understand that LLMs (as well as human brains) are not just autoregressive models, but nonlinear autoregressive models. Put a slightly different way, you can describe LLMs as autoregressive, but only by taking liberties with the classical definition of 'autoregressive.' The human mind is not, like ChatGPT and its ilk, a lumbering statistical engine for pattern matching, gorging on hun…
Author here and I welcome the feedback, but I don't really understand your point. My post is clearly not dismissive of efforts to make LLMs reason using CoT prompting techniques and post-training, and I think such efforts are even mentioned. The model remains autoregressive either way, and this reasoning is not some kind of magic that makes them behave differently - these improvements only make them perform (much) be…
Not saying that our current approaches will lead to intelligence. No one can know.
It could very well be that the internal mechanism of our thought has an auto-regressive reasoning component.
With the full system effectively "combining" short term memory (what just happened) and "pruned" long-term memory (what relevant things i know from the past) and pushing that into a RAW autoregressive reasoning component.
It is also possible that another specialized auto regressive reasoning component is driving the "prune" and "combine" operations. This whole system could be solely represented in the larger network.
The argument that "intelligence cannot be auto-regressive" seems to be without basis to me.
> there is strong evidence that not all thinking is linguistic or sequential.
It is possible that a system wrapping a core auto-regressive reasoner can produce non-sequential thinking - even if you don't allow for weight updates.
Re: Some thoughts on autoregressive models
#7Similar arguments to LeCun.
People are going to keep saying this about autoregressive models, how small errors accumulate and can't be corrected, while we literally watch reasoning models say things like "oh that's not right, let me try a different approach".
To me, this is like people saying "well NAND gates clearly can't sort things so I don't see how a computer could".
Large transformers can clearly learn very complex behavior, and the limits of that are not obvious from their low level building blocks or training paradigms.
Re: Some thoughts on autoregressive models
#8Similar arguments to LeCun. People are going to keep saying this about autoregressive models, how small errors accumulate and can't be corrected, while we literally watch reasoning models say things like "oh that's not right, let me try a different approach". To me, this is like people saying "well NAND gates clearly can't sort things so I don't see how a computer could". Large transformers can clearly learn very com…
I think recurrent training approaches like those discussed in COCONUT and similar papers show promising potential. As these techniques mature, models could eventually leverage their recurrent architecture to perform tasks requiring precise sequential reasoning, like odd/even bit counting that current architectures struggle with.
Re: Some thoughts on autoregressive models
#9Is anyone aware of a formalization of the idea that to get “symbols” out of fuzzy probability distributions one needs distributions whose value goes exactly to zero over some regions of the domain? I.e. Gaussian mixtures won’t cut it. And they will need very high Fourier frequencies.
I have the gut feeling that until a model allows for a small probability that 2x3 is 7, there will always be hallucinations. Probabilities need to be clamped to zero to emulate symbolic behaviour.
Re: Some thoughts on autoregressive models
#10>Most generative AI models nowadays are autoregressive.
There are also diffusion based models which don't rely on next token prediction.