Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Some thoughts on autoregressive models

wonderfall.dev

Post reply on HN
Print view View original

31–40 of 60 posts

Re: Some thoughts on autoregressive models

#31

Post by Wonderfall »

mxwsn wrote: post #14

> But what is the original purpose of AI research? I will speak for myself here, but I know many other AI researchers will say the same: the ultimate goal is to understand how humans think. And we think the best (or the funniest) way to understand how humans think is to try to recreate it. Eh. To riff on Dijkstra, this is like submarine engineers saying their ultimate goal is to understand how fish swim.

I come from a medical science background, where I studied the brain from a "traditional" neuroscience perspective (biology, pathology, anatomy, psychology and whatnot). That the best way is actually to try to recreate it is honestly how I feel whenever I read about AI advancements where the clear goal is to achieve/surpass human intelligence, something we don't fully understand yet.

“What I cannot create, I do not understand.” someone clever once said.

Re: Some thoughts on autoregressive models

#32

Post by _heimdall »

TeMPOraL wrote: post #18
dartos wrote: post #15

Earlier quoted context omitted.

> while we literally watch reasoning models say things like "oh that's not right, let me try a different approach". Not saying I disagree with your premise that errors can’t be corrected by using more and more tokens, but this argument is weird to me. The model isn’t intentionally generating text. The kinds of “oh let me try a different approach” lines I see are often followed by the same approach just taken. I would…

> Just because a model generates text doesn’t mean that the text actually represents anything at all, let alone a reflection of an internal process. What does it represent then? What are all these billion weights for? It's not a bag full of NULLs that just pulls next words from a look-up table. Obviously there is some kind of internal process. Also I don't get why people ignore the temporal aspect. Humans too generat…

The text represents a prediction of how a human may respond, one word(ish) at a time, that's it.

With "reasoning" models, the reasoning layer is basically another LLM instructed to specifically predict how a human may respond to the underlying LLM's answer, fake prompt engineering if you will.

There of course is some kind of internal process, but we can't prove any kind of reasoning. We ask a question, the main LLM responds, and we see how the reasoning layer LLM itself responds to that.

Re: Some thoughts on autoregressive models

#33

Post by dartos »

TeMPOraL wrote: post #18
dartos wrote: post #15

Earlier quoted context omitted.

> while we literally watch reasoning models say things like "oh that's not right, let me try a different approach". Not saying I disagree with your premise that errors can’t be corrected by using more and more tokens, but this argument is weird to me. The model isn’t intentionally generating text. The kinds of “oh let me try a different approach” lines I see are often followed by the same approach just taken. I would…

> Just because a model generates text doesn’t mean that the text actually represents anything at all, let alone a reflection of an internal process. What does it represent then? What are all these billion weights for? It's not a bag full of NULLs that just pulls next words from a look-up table. Obviously there is some kind of internal process. Also I don't get why people ignore the temporal aspect. Humans too generat…

> Humans too generate thoughts in sequence,

You don’t know this. I don’t feel like I generate thoughts in sequence, for me it feels hierarchical.

> can't arbitrarily mutate what came before

Uhh… what?

Do you remember your memories as a child? Or what you ate for breakfast 3 weeks ago?

Have you ever misremembered an event or half remembered a solution to a problem?

The information in human minds are entirely mutable. They are not like computers…

> It's not a bag full of NULLs that just pulls next words from a look-up table.

Funny enough, the attention mechanism that’s popular right now is effectively lots and lots of stacked look up tables. That’s how it’s taught as well (what with the Q K and V)

Tho I don’t think that’s a requirement for LLMs in general.

I find a lot of people who half understand cognition and understand computing look at LLMs and work backwards to convince themselves that it’s “thinking” or doing more cognitive functions like we humans do. It’s personification bias.

Re: Some thoughts on autoregressive models

#34

Post by PartiallyTyped »

fergal_reid wrote: post #7

Similar arguments to LeCun. People are going to keep saying this about autoregressive models, how small errors accumulate and can't be corrected, while we literally watch reasoning models say things like "oh that's not right, let me try a different approach". To me, this is like people saying "well NAND gates clearly can't sort things so I don't see how a computer could". Large transformers can clearly learn very com…

I'd argue that humans are by definition autoregressive "models", and we can change our minds mid thought as we process logical arguments. The issue around small errors accumulating makes sense if there is no sense of evaluation and recovery, but clearly, both evaluation and recovery is done.

Of course, this usually requires the human to have some sense of humility and admit their mistakes.

I wonder, what if we trained more models with data that self-heals or recovers mid sentence?

Re: Some thoughts on autoregressive models

#35

Post by _0ffh »

_heimdall wrote: post #32
TeMPOraL wrote: post #18

Earlier quoted context omitted.

> Just because a model generates text doesn’t mean that the text actually represents anything at all, let alone a reflection of an internal process. What does it represent then? What are all these billion weights for? It's not a bag full of NULLs that just pulls next words from a look-up table. Obviously there is some kind of internal process. Also I don't get why people ignore the temporal aspect. Humans too generat…

The text represents a prediction of how a human may respond, one word(ish) at a time, that's it. With "reasoning" models, the reasoning layer is basically another LLM instructed to specifically predict how a human may respond to the underlying LLM's answer, fake prompt engineering if you will. There of course is some kind of internal process, but we can't prove any kind of reasoning. We ask a question, the main LLM r…

Please don't confuse people with wrong information, the reasoning part in reasoning models is the exact same LLM that produces the final answer. For example o1 uses special "thinking" tokens to demarcate between reasoning and answer sections of it's output.

Re: Some thoughts on autoregressive models

#36

Post by aoeusnth1 »

I think the author is projecting significantly when he says the goal of AI researchers is to understand and replicate how humans think. If you start from that wrong assumption of course it looks silly for them to be doing anything other than neuroscience research, the author's field.

It's like saying the stockfish developers should stop researching mixed NN and search methods because they don't understand how humans play chess yet.

Re: Some thoughts on autoregressive models

#37

Post by UncleEntity »

Wonderfall wrote: post #31
mxwsn wrote: post #14

> But what is the original purpose of AI research? I will speak for myself here, but I know many other AI researchers will say the same: the ultimate goal is to understand how humans think. And we think the best (or the funniest) way to understand how humans think is to try to recreate it. Eh. To riff on Dijkstra, this is like submarine engineers saying their ultimate goal is to understand how fish swim.

I come from a medical science background, where I studied the brain from a "traditional" neuroscience perspective (biology, pathology, anatomy, psychology and whatnot). That the best way is actually to try to recreate it is honestly how I feel whenever I read about AI advancements where the clear goal is to achieve/surpass human intelligence, something we don't fully understand yet. “What I cannot create, I do not un…

It doesn't really follow that we (humans) have to replicate how we (humans) gained intelligence, there very well could be a shortcut that doesn't involve millions of years of getting eaten by tigers.

Re: Some thoughts on autoregressive models

#38

Post by Wonderfall »

aoeusnth1 wrote: post #36

I think the author is projecting significantly when he says the goal of AI researchers is to understand and replicate how humans think. If you start from that wrong assumption of course it looks silly for them to be doing anything other than neuroscience research, the author's field. It's like saying the stockfish developers should stop researching mixed NN and search methods because they don't understand how humans…

This is mainly a misunderstanding due to the way I phrased it. This is what I think. I know for a fact that is the case for other AI researchers having watched many conferences - "all of them" is not what I meant (I wrote "many other") and we certainly need people to approach problems from different perspectives and backgrounds, since they will benefit from each other in the end. Not going to lie I'm a bit disappointed to see these kind of comments.

Re: Some thoughts on autoregressive models

#39

Post by aoeusnth1 »

Wonderfall wrote: post #38
aoeusnth1 wrote: post #36

I think the author is projecting significantly when he says the goal of AI researchers is to understand and replicate how humans think. If you start from that wrong assumption of course it looks silly for them to be doing anything other than neuroscience research, the author's field. It's like saying the stockfish developers should stop researching mixed NN and search methods because they don't understand how humans…

This is mainly a misunderstanding due to the way I phrased it. This is what I think. I know for a fact that is the case for other AI researchers having watched many conferences - "all of them" is not what I meant (I wrote "many other") and we certainly need people to approach problems from different perspectives and backgrounds, since they will benefit from each other in the end. Not going to lie I'm a bit disappoint…

Fair enough about your motivation, however you also to further in saying that the best way to achieve and exceed human intelligence is to first understand it. That didn't pan out for chess, it hasn't contributed much to our current SOTA approaches to many other problems where LLMs are king, and I'm not sure why neuroscientists are so confident in some future where their field is the key to intelligence when their track record of breakthroughs is so poor.

Re: Some thoughts on autoregressive models

#40

Post by Wonderfall »

aoeusnth1 wrote: post #39
Wonderfall wrote: post #38

Earlier quoted context omitted.

This is mainly a misunderstanding due to the way I phrased it. This is what I think. I know for a fact that is the case for other AI researchers having watched many conferences - "all of them" is not what I meant (I wrote "many other") and we certainly need people to approach problems from different perspectives and backgrounds, since they will benefit from each other in the end. Not going to lie I'm a bit disappoint…

Fair enough about your motivation, however you also to further in saying that the best way to achieve and exceed human intelligence is to first understand it. That didn't pan out for chess, it hasn't contributed much to our current SOTA approaches to many other problems where LLMs are king, and I'm not sure why neuroscientists are so confident in some future where their field is the key to intelligence when their tra…

I admit my phrasing was poor there, and I got too excited. I will clarify since I don't really disagree with you or what others said (claiming it's the best way is an overstatement).

Well, one could say that neural networks pioneers modeled their ideas on simplified brain structures representations. Modern neural networks have little in common with an actual biological brain, however, the inspiration remains there (even for modern NNs like CNNs). I recall the intent was there too, originally: providing a framework to study biological cognition in the 50s. Then it evolved to become a new paradigm in computer science so that we have programs able to learn and adapt for problems that are formally too complicated for deterministic solutions.

Post reply on HN