Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A non-anthropomorphized view of LLMs

addxorrol.blogspot.com

Post reply on HN
Print view View original

171–180 of 432 posts

Re: A non-anthropomorphized view of LLMs

#171

Post by Timwi »

seadan83 wrote: post #130
ants_everywhere wrote: post #21

Earlier quoted context omitted.

> My question: how do we know that this is not similar to how human brains work. It is similar to how human brains operate. LLMs are the (current) culmination of at least 80 years of research on building computational models of the human brain.

> It is similar to how human brains operate. Is it? Do we know how human brains operate? We know the basic architecture of them, so we have a map, but we don't know the details. "The cellular biology of brains is relatively well-understood, but neuroscientists have not yet generated a theory explaining how brains work. Explanations of how neurons collectively operate to produce what brains can do are tentative and in…

> > It is similar to how human brains operate.

> Is it?

This is just a semantic debate on what counts as “similar”. It's possible to disagree on this point despite agreeing on everything relating to how LLMs and human brains work.

Re: A non-anthropomorphized view of LLMs

#172

Post by latexr »

lostmsu wrote: post #105
CharlesW wrote: post #10

Earlier quoted context omitted.

> The flip side of all this is of course the idea that there is still something emergent, unplanned, and mind- like. For people who have only a surface-level understanding of how they work, yes. A nuance of Clarke's law that "any sufficiently advanced technology is indistinguishable from magic" is that the bar is different for everybody and the depth of their understanding of the technology in question. That bar is s…

Nah, as a person that knows in detail how LLMs work with probably unique alternative perspective in addition to the commonplace one, I found any claims of them not having emergent behaviors to be of the same fallacy as claiming that crows can't be black because they have DNA of a bird.

> the same fallacy as claiming that crows can't be black because they have DNA of a bird.

What fallacy is that? I’m a fan of logical fallacies and never heard that claim before nor am I finding any reference with a quick search.

Re: A non-anthropomorphized view of LLMs

#173

Post by quonn »

jillesvangurp wrote: post #158

People anthropomorphize just about anything around them. People talk about inanimate objects like they are persons. Ships, cars, etc. And of course animals are well in scope for this as well, even the ones that show little to no signs of being able to reciprocate the relationship (e.g. an ant). People talk to their plants even. It's what we do. We can't help ourselves. There's nothing crazy about it and most people a…

> LLMs are not conscious because unlike human brains they don't learn or adapt (yet).

That's neither a necessary nor sufficient condition.

In order to be conscious, learning may not be needed, but a perception of the passing of time may be needed which may require some short-term memory. People with severe dementia often can't even remember the start of a sentence they are reading, they can't learn, but they are certainly conscious because they have just enough short-term memory.

And learning is not sufficient either. Consciousness is about being a subject, about having a subjective experience of "being there" and just learning by itself does not create this experience. There is plenty of software that can do some form of real-time learning but it doesn't have a subjective experience.

Re: A non-anthropomorphized view of LLMs

#174

Post by barrkel »

gpm wrote: post #86
barrkel wrote: post #32

Earlier quoted context omitted.

There is hidden state as plain as day merely in the fact that logits for token prediction exist. The selected token doesn't give you information about how probable other tokens were. That information, that state which is recalculated in autoregression, is hidden. It's not exposed. You can't see it in the text produced by the model. There is plenty of state not visible when an LLM starts a sentence that only becomes s…

The LLM does not "have" a plan. Arguably there's reason to believe it comes up with a plan when it is computing token propabilities, but it does not store it between tokens. I.e. it doesn't possess or "have" it. It simply comes up with a plan, emits a token, and entirely throws all its intermediate thoughts (including any plan) to start again from scratch on the next token.

I believe saying the LLM has a plan is a useful anthropomorphism for the fact that it does have hidden state that predicts future tokens, and this state conditions the tokens it produces earlier in the stream.

Re: A non-anthropomorphized view of LLMs

#175

Post by barrkel »

gugagore wrote: post #39
barrkel wrote: post #27

Earlier quoted context omitted.

Hidden state in the form of the activation heads, intermediate activations and so on. Logically, in autoregression these are recalculated every time you run the sequence to predict the next token. The point is, the entire NN state isn't output for each token. There is lots of hidden state that goes into selecting that token and the token isn't a full representation of that information.

That's not what "state" means, typically. The "state of mind" you're in affects the words you say in response to something. Intermediate activations isn't "state". The tokens that have already been generated, along with the fixed weights, is the only data that affects the next tokens.

Sure it's state. It logically evolves stepwise per token generation. It encapsulates the LLM's understanding of the text so far so it can predict the next token. That it is merely a fixed function of other data isn't interesting or useful to say.

All deterministic programs are fixed functions of program code, inputs and computation steps, but we don't say that they don't have state. It's not a useful distinction for communicating among humans.

Re: A non-anthropomorphized view of LLMs

#176

Post by latexr »

mewpmewp2 wrote: post #16

My question: how do we know that this is not similar to how human brains work. What seems intuitively logical to me is that we have brains evolved through evolutionary process via random mutations yielding in a structure that has its own evolutionary reward based algorithms designing it yielding a structure that at any point is trying to predict next actions to maximise survival/procreation, of course with a lot of s…

> how do we know that this is not similar to how human brains work.

Do you forget every conversation as soon as you have them? When speaking to another person, do they need to repeat literally everything they said and that you said, in order, for you to retain context?

If not, your brain does not work like an LLM. If yes, please stop what you’re doing right now and call a doctor with this knowledge. I hope Memento (2000) was part of your training data, you’re going to need it.

Re: A non-anthropomorphized view of LLMs

#178

Post by rtadred »

ninetyninenine wrote: post #125

[flagged]

https://rentry.co/2re4t2kx

This is what I got pasting the blog post in a prompt asking deepseep to write a reply in a stereotypical hackernews manner.

You are about as useful as a LLM as it can replicate your shallow memetics worthless train of thought.

Re: A non-anthropomorphized view of LLMs

#179

Post by barrkel »

NiloCK wrote: post #85
gugagore wrote: post #39

Earlier quoted context omitted.

That's not what "state" means, typically. The "state of mind" you're in affects the words you say in response to something. Intermediate activations isn't "state". The tokens that have already been generated, along with the fixed weights, is the only data that affects the next tokens.

Plus a randomness seed. The 'hidden state' being referred to here is essentially the "what might have been" had the dice rolls gone differently (eg, been seeded differently).

No, that's not quite what I mean. I used the logits in another reply to point out that there is data specific to the generation process that is not available from the tokens, but there's also the network activations adding up to that state.

Processing tokens is a bit like ticks in a CPU, where the model weights are the program code, and tokens are both input and output. The computation that occurs logically retains concepts and plans over multiple token generation steps.

That it is fully deterministic is no more interesting than saying a variable in a single threaded program is not state because you can recompute its value by replaying the program with the same inputs. It seems to me that this uninteresting distinction is the GP's issue.

Re: A non-anthropomorphized view of LLMs

#180

Post by whilenot-dev »

raincole wrote: post #159
Al-Khwarizmi wrote: post #117

I have the technical knowledge to know how LLMs work, but I still find it pointless to not anthropomorphize, at least to an extent. The language of "generator that stochastically produces the next word" is just not very useful when you're talking about, e.g., an LLM that is answering complex world modeling questions or generating a creative story. It's at the wrong level of abstraction, just as if you were discussing…

I've said that before: we have been anthropomorphizing computers since the dawn of information age. - Read and write - Behaviors that separate humans from animals. Now used for input and output. - Server and client - Human social roles. Now used to describe network architecture. - Editor - Human occupation. Now a kind of software. - Computer - Human occupation! And I'm sure people referred their cars and ships as 'he…

I'm not convinced... we use these terms to assign roles, yes, but these roles describe a utility or assign a responsibility. That isn't anthropomorphizing anything, but it rather describes the usage of an inanimate object as tool for us humans and seems in line with history.

What's the utility or the responsibility of AI, what's its usage as tool? If you'd ask me it should be closer to serving insights than "reasoning thoughts".

Post reply on HN