Live data from Hacker News

Emotion concepts and their function in a large language model

anthropic.com

211–212 of 212 posts

Re: Emotion concepts and their function in a large language model

#211

Earlier quoted context omitted.

I think you missed some of the point. If you say "Display information A using B format" but the model doesn't know A then you will get a more negative "emotional" response (e.g. desparation "I don't know this, but I am supposed to display it, I will just make something up") Taking that into account allows you to get better responses from the tool. It's not sentient, but it also is more complicated than bytecode.

Hmm, maybe. Though my initial reaction is the response isn't "emotional". An LLM isn't capable of emotion. Sure it's capable of assessing a quantitative score of sentiment to words/phrases...though that's not the same as an actual emotion. If the tool being used generates fantastical fiction that isn't supported by factual data or verifiable systems, then eventually that falsehood will bubble to the surface; whether…

Then substitute another word for emotion that fit better for you, but I would much rather take into account what this article is saying (e.g. using this string of text is likely to give you hallucinations. see my example) then have production bugs.

If it is your job to get deterministic results, this is a tool in the toolbox to do that and your original approach has been shown (see the results in this paper) to generate worse outcomes.

Re: Emotion concepts and their function in a large language model

#212

Earlier quoted context omitted.

If you read carefully my point is not about the external behavior of the LLM. It is the black box aspect of the LLM. The sheer complexity of the pure function is not something we can understand even though the high level structure is a feed forward network the core algorithm is in actuality encoded by weights. Yes there are complex functions besides LLMs that we don’t understand but those functions usually aren’t com…

I might have not explained it clearly, but my position is not what you have said. I agree with you that in principle it will be possible to design an artificial automaton that will have something equivalent with human emotions (though I do not believe that it makes sense to attempt to design such a system). However, I do not believe that an LLM is such a thing, because the training algorithm just ensures that an LLM…

>However, I do not believe that an LLM is such a thing, because the training algorithm just ensures that an LLM will mimic whatever is recorded in the training inputs, with or without human emotions in them.

This does not mean the underlying mechanism does not involve emotions. The logic does not follow. If you train a model to find a solution, it often in actuality becomes a models that finds the solution. It's not always the case that the model becomes a model that mimics finding the solution.

It's the same thing with emotions. You train it to output emotions, it is not always the case that the output of emotions is just a mimic of the emotions. We don't actually know.

>Regarding human emotions, they are recorded in a static way in a book or in a movie, but we do not say that the book or the movie has human emotions itself.

But the LLM is not not a book. It is something 'else'... an alien intelligence that emerges from training it on books. Your analogy does not follow.

>With an LLM, the behavior is much more complex, because it does not just play a sequential recording of human emotions, but it can combine them in various way, while responding to various stimuli that are similar to those that had elicited emotions in the training texts.

You don't know this. It may feel the emotion in it's own way. You're making a careless statement here without proof, knowledge or evidence.

>But regardless of this behavioral complexity, the human emotions are not generated somehow intrinsically by the LLM, but they correspond to those previously recorded in the texts used for training, so they just mimic humans.

Again you don't know this. You can't even formally define what a human emotion is which is a flaw on top of the fact that the black box nature of the LLM prevents you from understanding what an LLM id doing or "feeling".

Let's say human emotions produces a certain configuration of patterns of action potentials across the brain and we have sufficient sophistication to categorize these patterns in the same way we can categorize all the complex possibility of say rodents or fruit. If we had that WE still wouldn't know if the LLM felt emotions SIMPLY because it is a black box. It may be the thing we trained in order to "mimic" human emotions actually produces the same configuration pattern of numerical signals flowing through the feed forward network that fits in the "category" of an emotion.

One possible training outcome to meeting the requirement of "mimicking" emotions is to actually produce the emotion itself in order to mimic it.

Post reply on HN