Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A non-anthropomorphized view of LLMs

addxorrol.blogspot.com

Post reply on HN
Print view View original

281–290 of 432 posts

Re: A non-anthropomorphized view of LLMs

#281

Post by [deleted] »

barrkel wrote: post #174
gpm wrote: post #86

Earlier quoted context omitted.

The LLM does not "have" a plan. Arguably there's reason to believe it comes up with a plan when it is computing token propabilities, but it does not store it between tokens. I.e. it doesn't possess or "have" it. It simply comes up with a plan, emits a token, and entirely throws all its intermediate thoughts (including any plan) to start again from scratch on the next token.

I believe saying the LLM has a plan is a useful anthropomorphism for the fact that it does have hidden state that predicts future tokens, and this state conditions the tokens it produces earlier in the stream.

[deleted]

Re: A non-anthropomorphized view of LLMs

#282

Post by andy12_ »

ACCount36 wrote: post #144
lostmsu wrote: post #102

Earlier quoted context omitted.

This is wrong, intermediate activations are preserved when going forward.

Within a single forward pass, but not from one emitted token to another.

What? No. The intermediate hidden states are preserved from one token to another. A token that is 100k tokens into the future will be able to look into the information of the present token's hidden state through the attention mechanism. This is why the KV cache is so big.

Re: A non-anthropomorphized view of LLMs

#283

Post by sdenton4 »

losvedir wrote: post #265
grey-area wrote: post #120

Earlier quoted context omitted.

On the contrary, anthropomorphism IMO is the main problem with narratives around LLMs - people are genuinely talking about them thinking and reasoning when they are doing nothing of that sort (actively encouraged by the companies selling them) and it is completely distorting discussions on their use and perceptions of their utility.

Well "reasoning" refers to Chain-of-Thought and if you look at the generated prompts it's not hard to see why it's called that. That said, it's fascinating to me that it works (and empirically, it does work; a reasoning model generating tens of thousands of tokens while working out the problem does produce better results). I wish I knew why. A priori I wouldn't have expected it, since there's no new input. That means…

CoT gives the model more time to think and process the inputs it has. To give an extreme example, suppose you are using next token prediction to answer 'Is P==NP?' The tiny number of input tokens means that there's a tiny amount of compute to dedicate to producing an answer. A scratchpad allows us to break free of the short-inputs problem.

Meanwhile, things can happen in the latent representation which aren't reflected in the intermediate outputs. You could, instead of using CoT, say "Write a recipe for a vegetarian chile, along with a lengthy biographical story relating to the recipe. Afterwards, I will ask you again about my original question." And the latents can still help model the primary problem, yielding a better answer than you would have gotten with the short input alone.

Along these lines, I believe there are chain of thought studies which find that the content of the intermediate outputs don't actually matter all that much...

Re: A non-anthropomorphized view of LLMs

#284

Post by jibal »

pmg101 wrote: post #261
Al-Khwarizmi wrote: post #117

I have the technical knowledge to know how LLMs work, but I still find it pointless to not anthropomorphize, at least to an extent. The language of "generator that stochastically produces the next word" is just not very useful when you're talking about, e.g., an LLM that is answering complex world modeling questions or generating a creative story. It's at the wrong level of abstraction, just as if you were discussing…

I remember Dawkins talking about the "intentional stance" when discussing genes in The Selfish Gene. It's flat wrong to describe genes as having any agency. However it's a useful and easily understood shorthand to describe them in that way rather than every time use the full formulation of "organisms who tend to possess these genes tend towards these behaviours." Sometimes to help our brains reach a higher level of a…

The intentional stance was Daniel Dennett's creation and a major part of his life's work. There are actually (exactly) three stances in his model: the physical stance, the design stance, and the intentional stance.

https://en.wikipedia.org/wiki/Intentional_stance

I think the design stance is appropriate for understanding and predicting LLM behavior, and the intentional stance is not.

Re: A non-anthropomorphized view of LLMs

#285

Post by thinkmassive »

LeonardoTolstoy wrote: post #192
flir wrote: post #154

Earlier quoted context omitted.

Agreeing with you, this is a "can a submarine swim" problem IMO. We need a new word for what LLMs are doing. Calling it "thinking" is stretching the word to breaking point, but "selecting the next word based on a complex statistical model" doesn't begin to capture what they're capable of. Maybe it's cog-nition (emphasis on the cog).

What does a submarine do? Submarine? I suppose you "drive" a submarine which is getting to the idea: submarines don't swim because ultimately they are "driven"? I guess the issue is we don't make up a new word for what submarines do, we just don't use human words. I think the above poster gets a little distracted by suggesting the models are creative which itself is disputed. Perhaps a better term, like above, would…

GenAI _generates_ output

Re: A non-anthropomorphized view of LLMs

#286

Post by shakadak »

losvedir wrote: post #265
grey-area wrote: post #120

Earlier quoted context omitted.

On the contrary, anthropomorphism IMO is the main problem with narratives around LLMs - people are genuinely talking about them thinking and reasoning when they are doing nothing of that sort (actively encouraged by the companies selling them) and it is completely distorting discussions on their use and perceptions of their utility.

Well "reasoning" refers to Chain-of-Thought and if you look at the generated prompts it's not hard to see why it's called that. That said, it's fascinating to me that it works (and empirically, it does work; a reasoning model generating tens of thousands of tokens while working out the problem does produce better results). I wish I knew why. A priori I wouldn't have expected it, since there's no new input. That means…

> I don't see why it couldn't just one shot it without all the reasoning.

That's reminding me of deep neural networks where single layer networks could achieve the same results, but the layer would have to be excessively large. Maybe we're re-using the same kind of improvement, scaling in length instead of width because of our computation limitations ?

Re: A non-anthropomorphized view of LLMs

#287

Post by naasking »

CharlesW wrote: post #10
szvsw wrote: post #3

So the author’s core view is ultimately a Searle-like view: a computational, functional, syntactic rules based system cannot reproduce a mind. Plenty of people will agree, plenty of people will disagree, and the answer is probably unknowable and just comes down to whatever axioms you subscribe to in re: consciousness. The author largely takes the view that it is more productive for us to ignore any anthropomorphic re…

> The flip side of all this is of course the idea that there is still something emergent, unplanned, and mind- like. For people who have only a surface-level understanding of how they work, yes. A nuance of Clarke's law that "any sufficiently advanced technology is indistinguishable from magic" is that the bar is different for everybody and the depth of their understanding of the technology in question. That bar is s…

> For people who have only a surface-level understanding of how they work, yes.

This is too dismissive because it's based on an assumption that we have a sufficiently accurate mechanistic model of the brain that we can know when something is or is not mind-like. This just isn't the case.

Re: A non-anthropomorphized view of LLMs

#288

Post by dotancohen »

JimDabell wrote: post #227
flir wrote: post #154

Earlier quoted context omitted.

Agreeing with you, this is a "can a submarine swim" problem IMO. We need a new word for what LLMs are doing. Calling it "thinking" is stretching the word to breaking point, but "selecting the next word based on a complex statistical model" doesn't begin to capture what they're capable of. Maybe it's cog-nition (emphasis on the cog).

> this is a "can a submarine swim" problem IMO. We need a new word for what LLMs are doing. Why? A plane is not a fly and does not stay aloft like a fly, yet we describe what it does as flying despite the fact that it does not flap its wings. What are the downsides we encounter that are caused by using the word “fly” to describe a plane travelling through the air?

For what it's worth, in my language the motion of birds and the motion of aircraft _are_ two different words.

Re: A non-anthropomorphized view of LLMs

#289

Post by jibal »

CuriousSkeptic wrote: post #193
tempfile wrote: post #184

Earlier quoted context omitted.

The "point" of not anthropomorphizing is to refrain from judgement until a more solid abstraction appears. The problem with explaining LLMs in terms of human behaviour is that, while we don't clearly understand what the LLM is doing, we understand human cognition even less! There is literally no predictive power in the abstraction "The LLM is thinking like I am thinking". It gives you no mechanism to evaluate what ta…

> Why don't LLMs get frustrated with you if you ask them the same question repeatedly? To be fair, I have had a strong sense of Gemini in particular becoming a lot more frustrated with me than GPT or Claude. Yesterday I had it ensuring me that it was doing a great job, it was just me not understanding the challenge but it would break it down step by step just to make it obvious to me (only to repeat the same errors,…

Point out to an LLM that it has no mental states and thus isn't capable of being frustrated (or glad that your program works or hoping that it will, etc. ... I call them out whenever they ascribe emotions to themselves) and they will confirm that ... you can coax from them quite detailed explanations of why and how it's an illusion.

Of course they will quickly revert to self-anthropomorphizing language, even after promising that they won't ... because they are just pattern matchers producing the sort of responses that conforms to the training data, not cognitive agents capable of making or keeping promises. It's an illusion.

Re: A non-anthropomorphized view of LLMs

#290

Post by mewpmewp2 »

latexr wrote: post #176
mewpmewp2 wrote: post #16

My question: how do we know that this is not similar to how human brains work. What seems intuitively logical to me is that we have brains evolved through evolutionary process via random mutations yielding in a structure that has its own evolutionary reward based algorithms designing it yielding a structure that at any point is trying to predict next actions to maximise survival/procreation, of course with a lot of s…

> how do we know that this is not similar to how human brains work. Do you forget every conversation as soon as you have them? When speaking to another person, do they need to repeat literally everything they said and that you said, in order, for you to retain context? If not, your brain does not work like an LLM. If yes, please stop what you’re doing right now and call a doctor with this knowledge. I hope Memento (2…

Knowledge of every conversation must be some form of state in our minds, just like for LLMs it could be something retrieved from a database, no? I don't think information storing or retrieval is necessarily the most important achievements here in the first place. It's the emergent abilities that you wouldn't have expected to occur.
Post reply on HN