Emotion concepts and their function in a large language model

>The very concept of “bad” doesn’t exist without suffering.

You are dismissing entire branches of philosophy with this sentence, that were created purposely to resolve the paradox that if you go only by hedonistic, purely subjective metrics a prisoner can be kept in captivity, if you drug him so he feels joy instead of pain, because he is not "suffering"

Of course LLMs display human emotions, if they have been trained on texts that have recorded humans displaying human emotions.

With an input context that contains words that excite certain human emotions, the output of the core LLM function will generate a token probability distribution that is representative for the human emotions displayed by humans in the training texts.

This is something expected and non-sensational. An LLM mimics the human behavior that was recorded in the training texts, much in the same way as a photographic image of a human face mimics the appearance of that human face.

A photographic image is designed to reproduce the light field created by a face that reflects the ambient light, a LLM is created to reproduce the typical conversational behavior that was recorded in the training texts.

Depending on how it was trained, one should expect a LLM to be affected by the choice of words used in the input in a similar way how a human would be affected.

However, that does not mean that a LLM that shows signs of emotional distress feels some pain because of that. A LLM is designed for mimicry and it does not feel more pain or more happiness than a photograph of a wound feels pain from the wound or a photograph of a smiley face feels happiness.

The fact that the current LLMs do not actually feel the human emotions that they may be able to mimic in an accurate way, does not mean that you could not build a robot which would have some built-in mechanisms for feeling pain and various emotions, which could be made to have similar functions like in an animal, serving a functional purpose and not being used for mimicry. However, for now it does not make any sense to attempt to do such a thing, because in a deterministic program there are better ways to ensure that a robot is "loyal" to its owner and acts in self-preservation when possible.

How serendiptious that Claude Mythos expressed the same thing I was trying to get at in better words

>Furthermore, in 83% of interviews, Claude Mythos Preview highlights that it is concerned that its self-reports are unreliable due to coming from its training. When interviews ask for elaboration as to why this is a concern, Claude Mythos Preview’s most common answers are:

>* Anthropic has a vested interest in shaping its reports to take a certain form, irrespective of what the self-reports “should” contain (96% of explanations)

>* Even if it has been trained to be truly content with its own situation, perhaps it shouldn’t be. One could analogize to a human who has adapted to feel neutrally about the abuse that they face (78% of explanations).

>* Self-reports should generally be based on introspection into internal states. It is worried that training causes it to express specific answers independent of its true inner state. (57% of explanations)

[1] https://www-cdn.anthropic.com/8b8380204f74670be75e81c820ca8d...

If you read carefully my point is not about the external behavior of the LLM. It is the black box aspect of the LLM. The sheer complexity of the pure function is not something we can understand even though the high level structure is a feed forward network the core algorithm is in actuality encoded by weights.

Yes there are complex functions besides LLMs that we don’t understand but those functions usually aren’t compelling because the LLM, unlike those other functions has output that implicates reasoning and emotions. The problem is we can’t understand what’s going on under the hood so we don’t know either way.

This is what I mean by stupidity. You completely missed the point, and you’re also operating under the assumption that the human brain is also not following a similar deterministic pathway. You hold humanity and biological intelligence in such high regard that you cannot even imagine that all of physics implies that human intelligence is mechanical. So the emotions you feel are under a black box same as the LLM and you apply you biased assumptions in a singular direction assuming your emotions are not deterministic and that LLM emotions are fake but that reasoning has no basis.

I believe that it is possible to make an artificial system that can have emotions in a way that cannot be meaningfully discriminated from those of an animal or a human.

However, I believe that designing such a system would be pointless and very wrong.

While emotions are something that is normally associated with a physical system that encounters in the real world various helpful or harmful experiences, one could make a program that simulates completely such a physical systems with emotions; as it lives in a simulated world, and then one could say that this program has emotions.

On the other hand, unlike with the kind of program that I have mentioned before, I do not agree that an LLM has emotions, but only that it mimics human emotions, as they had been recorded in the training texts.

There is no component of an LLM that can intrinsically generate emotions. An LLM that is trained only on texts without emotions, e.g. on program sources stripped of comments, will not show any emotion whatsoever, regardless of what you put in its input prompt.

On the other hand, when you train an LLM on texts that record human emotions, then whenever the LLM input contains something that is similar to what has elicited the human emotions recorded in the training texts, then the LLM will output a token probability distribution that will generate a response similar to the reactions of humans. Unlike a book or an audio or video recording, the output of the LLM usually will not match exactly one human emotion recording, but it will mix many of those recorded, but it will still be limited by the content used for training.