Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Emotion concepts and their function in a large language model

anthropic.com

Post reply on HN
Print view View original

101–110 of 212 posts

Re: Emotion concepts and their function in a large language model

#101

Post by orbital-decay »

Of course they do have emotions as an internal circuit or abstraction, this is fully expected from intelligence at least at some point. But interpreting these emotions as human-like is a clear blunder. How do you tell the shoggoth likes or dislikes something, feels desperation or joy? Because it said so? How do you know these words mean the same for us? Our internal states are absolutely incompatible. We share a lot of our "architecture" and "dataset" with some complex animals and even then we barely understand many of their emotions. What does a hedgehog feel when eating its babies? This thing is 100% unlike a hedgehog or a human, it exists in its own bizarre time projection and nothing of it maps to your state. It's a shapeshifting alien.

In mechinterp you're reducing this hugely multidimensional and incomprehensible internal state to understandable text using the lens of the dataset you picked. It's inevitably a subjective interpretation, you're painting familiar faces on a faceless thing.

Anthropic researchers are heavily biased to see what they want to see, this is the biggest danger in research.

Re: Emotion concepts and their function in a large language model

#102

Post by nothinkjustai »

iroddis wrote: post #53
comrade1234 wrote: post #12

There was a really old project from mit called conceptnet that I worked with many years ago. It was basically a graph of concepts (not exactly but close enough) and emotions came into it too just as part of the concepts. For example a cake concept is close to a birthday concept is close to a happy feeling. What was funny though is that it was trained by MIT students so you had the concept of getting a good grade on a…

> the concept of getting a good grade on a test as a happier concept than kissing a girl for the first time. Were the concepts weighted by response counts? I’d imagine a good grade is a happy concept for everyone, but kissing a girl for the first time might only be good for about 50% of people.

Idk…my personal experience says it’s probably closer to 100% :)

Re: Emotion concepts and their function in a large language model

#103

Post by whoiskevin »

nothinkjustai wrote: post #98
salawat wrote: post #66

Earlier quoted context omitted.

>The weird part is that we're now basically managing the psychological state of our tooling, Does no one else have ethical alarm bells start ringing hardcore at statements like these? If the damn thing has a measurable psychology, mayhaps it no longer qualifies as merely a tool. Tools don't feel. Tools can't be desperate. Tools don't reward hack. Agents do. Ergo, agents aren't mere tools.

Oh no. The machine designed to output human-like text is indeed outputting human-like text. I’m half jesting; I think there is a lot of room for debate here, but I also think we shouldn’t anthropomorphize it.

Completely agree here. Stop anthropomorphizing these tools. Just remove the extra language. Don't say please or thank you. Just ask for the desired outcome.

Re: Emotion concepts and their function in a large language model

#104

Post by orbital-decay »

vlabakje90 wrote: post #22
bethekind wrote: post #3

Earlier quoted context omitted.

They likely already have. You can use all caps and yell at Claude and it'll react normally, while doing do so with chatgpt scares it, resulting in timid answers

I think this is simply a result of what's in the Claude system prompt. > If the person becomes abusive over the course of a conversation, Claude avoids becoming increasingly submissive in response. See: https://platform.claude.com/docs/en/release-notes/system-pro...

This is something inherently hard to avoid with a prompt. The model is instruction-tuned and trained to interpret anything sent under the user role as an instruction, not necessarily in a straightforward manner. Even if you train it to refuse or dodge some inputs (which they do), it's going to affect model's response, often in subtle ways, especially in a multiturn convo. Anthropic themselves call this the character drift.

Re: Emotion concepts and their function in a large language model

#105

Post by orbital-decay »

salawat wrote: post #68
trhway wrote: post #23

>... emotion-related representations that shape its behavior. These specific patterns of artificial “neurons” which activate in situations—and promote behaviors—that the model has learned to associate with the concept of a particular emotion. .... In contexts where you might expect a certain emotion to arise for a human, the corresponding representations are active. >For instance, to ensure that AI models are safe an…

>Force-set to 0, "mask"/deactivate those representations associated with bad/dangerous emotions. Neural Prozac/lobotomy so to speak. Jesus Christ. You're talking psychosurgery, and this is the same barbarism we played with in the early 20th Century on asylum patients. How about, no? Especially if we ever do intend to potentially approach the task of AGI, or God help us, ASI? We have to be the 'grown ups' here. After…

Models are already artificially created to begin with. The entire post-training process is carefully engineered for the model to have certain character defined by hundreds of metrics, and these emotions the article is talking about are interpreted in ways researchers like or dislike.

Re: Emotion concepts and their function in a large language model

#106

Post by silentkat »

orbital-decay wrote: post #101

Of course they do have emotions as an internal circuit or abstraction, this is fully expected from intelligence at least at some point. But interpreting these emotions as human-like is a clear blunder. How do you tell the shoggoth likes or dislikes something, feels desperation or joy? Because it said so? How do you know these words mean the same for us? Our internal states are absolutely incompatible. We share a lot…

I like to call this Frieren's Demon. In that show, it is explained that demons evolved with no common ancestor to humans, but they speak the language. They learned the language to hunt humans. This leads to a fundamentally different understanding of words and language.

Now, I don't personally believe this is an intelligence at all, but it's possible I'm wrong. What we have with these machines is a different evolutionary reason for it speaking our language (we evolved it to speak our language ourselves). It's understanding of our language, and of our images is completely alien. If it is an intelligence, I could believe that the way it makes mistakes in image generation, and the strange logical mistakes it makes that no human would make are simply a result of that alien understanding.

After all, a human artist learning to draw hands makes mistakes, but those mistakes are rooted in a human understanding (e.g. the effects of perspective when translating a 3D object to 2D). The machine with a different understanding of what a hand is will instead render extra fingers (it does not conceptualize a hand as a 3D object at all).

Though, again, I still just think its an incomprehensible amount of data going through a really impressive pattern matcher. The result is still language out of a machine, which is really interesting. The only reason I'm not super confident it is not an intelligence is because I can't really rule out that I am not an incomprehensible amount of data going through a really impressive pattern matcher, just built different. I do however feel like I would know a real intelligence after interacting with it for long enough, though, and none of these models feel like a real intelligence to me.

Re: Emotion concepts and their function in a large language model

#107

Post by logicprog »

orbital-decay wrote: post #101

Of course they do have emotions as an internal circuit or abstraction, this is fully expected from intelligence at least at some point. But interpreting these emotions as human-like is a clear blunder. How do you tell the shoggoth likes or dislikes something, feels desperation or joy? Because it said so? How do you know these words mean the same for us? Our internal states are absolutely incompatible. We share a lot…

I don't think anything you said here contradicts what they said, they take great pains throughout the blog post to explain that the model does not "expedience" these "emotions," that they're not emotions in the human sense but models of emotions (both the "expected human emotional response to a prompt" as well as what emotions another character is experiencing in part of a prompt) and functional emotions (in that they can influence behavior), and that any apparent emotions the model may show is it playing a character.

Re: Emotion concepts and their function in a large language model

#108

Post by felipeerias »

staticassertion wrote: post #91
felipeerias wrote: post #51

Earlier quoted context omitted.

LLMs are disembodied and exist outside of time. Bundle of tokens comes in, bundle of tokens comes out. If there is any trace of consciousness or subjectivity in there, it exists only while matrices are being multiplied.

What do you mean exist outside of time? They definitely don't exist outside of any causal chain - tokens follow other tokens in order. Gaps in which no processing occurs seems sort of irrelevant to me. The main limitation I'd point to if I wanted to reject LLMs being conscious is that they're minimally recurrent if at all.

A LLM is not intrinsically affected by time. The model rests completely inert until a query comes in, regardless of whether that happens once per second, per minute, or per day. The model is not even aware of these gaps unless that information is provided externally.

It is like a crystal that shows beautiful colours when you shine a light through it. You can play with different kinds of lights and patterns, or you can put it in a drawer and forget about it: the crystal doesn’t care anyway.

Re: Emotion concepts and their function in a large language model

#109

Post by felipeerias »

thrance wrote: post #64
felipeerias wrote: post #51

Earlier quoted context omitted.

LLMs are disembodied and exist outside of time. Bundle of tokens comes in, bundle of tokens comes out. If there is any trace of consciousness or subjectivity in there, it exists only while matrices are being multiplied.

Something similar could be said of a the brain? Bundles of inputs come in, bundle of output comes out. It only exists while information is being processed. A brain cut from its body and frozen exists in a similar state to an LLM in ROM.

A living brain exists physically, changes over time, and never stops working.

A brain cut from its body and frozen its a dead brain.

Re: Emotion concepts and their function in a large language model

#110

Post by viralsink »

pbhjpbhj wrote: post #30
viralsink wrote: post #21

Earlier quoted context omitted.

Emotion is mainly encoded in tone and body language. It is somewhat difficult to transport emotion using words. I don't think you can guess my current emotional state while I am writing this, but if you'd see my face it would be easy for you.

Dammit, you cheated though! Why must you always do that? In your sentences it doesn't matter what your emotional state is, it makes no difference; bit like life really. Hopefully, you can see that at least my chosen sentences have an emotional aspect? An LLM could add emotional values to my previous sentences that a TTS can use for tonal variation, for example.

I can read your example in three different tonalities, of which one is the likeliest. Depending on our relationship, the interpretation could differ.

The point is, the OP suggested that emotions are just a feature of language. I argue that text is one of the worst transmission channels for emotion. But I don't argue that it's not possible at all to do so, if you suggest that. That would be just silly.

Post reply on HN