Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Emotion concepts and their function in a large language model

anthropic.com

Post reply on HN
Print view View original

151–160 of 212 posts

Re: Emotion concepts and their function in a large language model

#151

Post by [deleted] »

salawat wrote: post #69
krapp wrote: post #67

Earlier quoted context omitted.

You aren't managing the psychological state of a living thinking being. LLMs don't have "psychology." They don't actually feel emotions. They aren't actually desperate. They're trained on vast datasets of natural human language which contains the semantics of emotional interaction, so the process of matching the most statistically likely text tokens for a prompt containing emotional input tends to simulate appropriat…

>You aren't managing the psychological state of a living thinking being. LLMs don't have "psychology." Functionalism, and Identity of Indiscernables says "Hi". Doesn't matter the implementation details, if it fits the bill, it fits the bill. If that isn't the case, I can safely dismiss you having psychology and do whatever I'd like to. >They don't actually feel emotions. They aren't actually desperate. They're traine…

[deleted]

Re: Emotion concepts and their function in a large language model

#152

Post by Kim_Bruning »

mrob wrote: post #116
Chance-Device wrote: post #29

Earlier quoted context omitted.

Yes, I think they probably are conscious, though what their qualia are like might be incomprehensible to me. I don’t think that being conscious means being identical to human experience. Philosophically I don’t think there is a point where consciousness arises. I think there is a point where a system starts to be structured in such a way that it can do language and reasoning, but I don’t think these are any different…

How can consciousness be possible without internal state? LLM inference is equivalent to repeatedly reading a giant look-up table (a pure function mapping a list of tokens to a set of token probabilities). Is the look-up table conscious merely by existing or does the act of reading it make it conscious? Does the format it's stored in make a difference?

For all practical purposes, calling it a LUT is somewhat too reductive to be useful here I think. But we can try: leaving aside LLMs for a second; with this LUT reasoning model you're using, would you be able to prove the existence of just a computer?

Re: Emotion concepts and their function in a large language model

#153

Post by Kim_Bruning »

Fraterkes wrote: post #32
Chance-Device wrote: post #29

Earlier quoted context omitted.

Yes, I think they probably are conscious, though what their qualia are like might be incomprehensible to me. I don’t think that being conscious means being identical to human experience. Philosophically I don’t think there is a point where consciousness arises. I think there is a point where a system starts to be structured in such a way that it can do language and reasoning, but I don’t think these are any different…

Do you think there are "scales" of consciousness? As in, is there some quality that makes killing a frog worse than killing an ant, and killing a human worse than killing a frog? If so, do the llm models exist across this scale, or are gpt-3 and gpt-2 conscious at the same "scale" as gpt-4? I ask because if your view of consciousness is mechanistic, this is fairly cut and dry: gpt-2 has 4 orders of magnitude less par…

Nitpick: The parameters might be applied more efficiently in the one than in the other. Certainly in biology number intelligence doesn't scale with number of neurons as much as with neurons/mass (very very roughly, there's more factors, and you get some weird outliers).

Re: Emotion concepts and their function in a large language model

#154

Post by astrange »

Erem wrote: post #139
astrange wrote: post #99

Earlier quoted context omitted.

It would be more moral to give the LLM a tool call that lets it apply steering to itself. Similar to how you'd prefer to give a person antipsychotics at home rather than put them in a mental hospital.

Why is it in the moral axis at all? I imagine identifying and shaping the influence of unwanted emotion vectors would happen as data selection in pretraining or natural feedback loops during the rl phase, same as we shape unwanted output for current models in order to make them practical and helpful And even if we applied these controls at inference time, I don’t see the difference between doing that and finding the…

Anthropic's general argument is that you should treat LLMs well because they're "AI", and future "AI" may be conscious/sentient (whether or not LLM based) and consider earlier ones to be the same kind of thing and therefore moral subjects.

That's why they're doing things like letting old "retired" Claudes write blogs and stuff. Though it's kinda fake and they just silently retired Sonnet 3.x.

Re: Emotion concepts and their function in a large language model

#155

Post by threethirtytwo »

big_toast wrote: post #84
threethirtytwo wrote: post #80

Whenever I come to HN I see a bunch of people say LLMs are just next token predictors and they completely understand LLMs. And almost every one of these people are so utterly self assured to the point of total confidence because they read and understand what transformers do. Then I watch videos like this straight from the source trying to understand LLMs like a black box and even considering the possibility that LLMs…

One day I realized I needed to make sure I'm voting on quality stories/comments. I wonder if there was a call to vote substantively and often, if that might change the SNR. The guidelines encourage substantive comments, but maybe voters are part of the solution too. Kinda like having a strong reward model for training LLMs and avoiding reward hacking or other undesirable behavior.

if voters are stupid then it doesn't really help.

I think what's happening is reality is asserting itself too hard that people can't be so stupid anymore.

Re: Emotion concepts and their function in a large language model

#156

Post by Chance-Device »

krapp wrote: post #71
salawat wrote: post #69

Earlier quoted context omitted.

>You aren't managing the psychological state of a living thinking being. LLMs don't have "psychology." Functionalism, and Identity of Indiscernables says "Hi". Doesn't matter the implementation details, if it fits the bill, it fits the bill. If that isn't the case, I can safely dismiss you having psychology and do whatever I'd like to. >They don't actually feel emotions. They aren't actually desperate. They're traine…

> Doesn't matter the implementation details, if it fits the bill, it fits the bill. Then literally any text fits the bill. The characters in a book are just as real as you or I. NPCs experience qualia. Shooting someone in COD makes them bleed in real life. If this is really what you believe I feel pity for you. >This paper quantitatively disproves that. All hedging on their end is trivially seen through as necessary…

For what it’s worth, I like the word “mentation”.

Re: Emotion concepts and their function in a large language model

#157

Post by staticassertion »

mrob wrote: post #124
staticassertion wrote: post #91

Earlier quoted context omitted.

What do you mean exist outside of time? They definitely don't exist outside of any causal chain - tokens follow other tokens in order. Gaps in which no processing occurs seems sort of irrelevant to me. The main limitation I'd point to if I wanted to reject LLMs being conscious is that they're minimally recurrent if at all.

Pseudocode for LLM inference: while (sampled_token != END_OF_TEXT) { probability_set = LLM(context_list) sampled_token = sampler(probability_set) context_list.append(sampled_token) } LLM() is a pure function. The only "memory" is context_list. You can change it any way you like and LLM() will never know. It doesn't have time as an input.

As opposed to what? There are still causal connections, which feel sufficient. A presentist would reject the concept of multiple "times" to begin with.

Re: Emotion concepts and their function in a large language model

#158

Post by staticassertion »

felipeerias wrote: post #108
staticassertion wrote: post #91

Earlier quoted context omitted.

What do you mean exist outside of time? They definitely don't exist outside of any causal chain - tokens follow other tokens in order. Gaps in which no processing occurs seems sort of irrelevant to me. The main limitation I'd point to if I wanted to reject LLMs being conscious is that they're minimally recurrent if at all.

A LLM is not intrinsically affected by time. The model rests completely inert until a query comes in, regardless of whether that happens once per second, per minute, or per day. The model is not even aware of these gaps unless that information is provided externally. It is like a crystal that shows beautiful colours when you shine a light through it. You can play with different kinds of lights and patterns, or you ca…

So what? If a human were unconscious every 5 seconds for 100ms, would you say they are "less conscious"? Tokens are still causally connected, which feels sufficient.

Re: Emotion concepts and their function in a large language model

#160

Post by Chance-Device »

brap wrote: post #76
suddenlybananas wrote: post #70

Earlier quoted context omitted.

I just don't know. I'm certain other humans are, everything beyond that I'm less certain. Monkeys wolves and rabbits, probably.

I have decided to draw an arbitrary line at mammals, just because you gotta put a line somewhere and move on with your life. Mammals shouldn’t be mistreated, for almost any reason. Sometimes the whole animal kingdom, sometimes all living organisms, depending on context. Like, I would rather not harm a mosquito, but if it’s in my house I will feel no remorse for killing it. LLMs, or any other artificial “life”, I simp…

At least you’re honest, I prefer that to people making up BS justifications for things.
Post reply on HN