Emotion concepts and their function in a large language model • Hacker News Board

Re: Emotion concepts and their function in a large language model

#111

Post by xg15 » Sat, Apr 04, 2026, 10:33 PM UTC

orbital-decay wrote: post #101

Of course they do have emotions as an internal circuit or abstraction, this is fully expected from intelligence at least at some point. But interpreting these emotions as human-like is a clear blunder. How do you tell the shoggoth likes or dislikes something, feels desperation or joy? Because it said so? How do you know these words mean the same for us? Our internal states are absolutely incompatible. We share a lot…

I think a counterargument would be parallel evolution: There are various examples in nature, where a certain feature evolved independently several times, without any genetic connection - from what I understand, we believe because the evolutionary pressures were similar.

One obvious example would be wings, where you have several different strategies - feathers, insect wings, bat-like wings, etc - that have similar functionality and employ the same physical principles, but are "implemented" vastly differently.

You have similar examples in brains, where e.g. corvids are capable of various cognitive feats that would involve the neocortex in human brains - only their brains don't have a neocortex. Instead they seem to use certain other brain regions for that, which don't have an equivalent in humans.

Nevertheless it's possible to communicate with corvids.

So this makes me wonder if a different "implementation" always necessarily means the results are incomparable.

In the interest of falsifiability, what behavior or internal structures in LLMs would be enough to be convincing that they are "real" emotions?

Re: Emotion concepts and their function in a large language model

#112

Post by mrob » Sat, Apr 04, 2026, 10:37 PM UTC

xg15 wrote: post #111

orbital-decay wrote: post #101

Of course they do have emotions as an internal circuit or abstraction, this is fully expected from intelligence at least at some point. But interpreting these emotions as human-like is a clear blunder. How do you tell the shoggoth likes or dislikes something, feels desperation or joy? Because it said so? How do you know these words mean the same for us? Our internal states are absolutely incompatible. We share a lot…

I think a counterargument would be parallel evolution: There are various examples in nature, where a certain feature evolved independently several times, without any genetic connection - from what I understand, we believe because the evolutionary pressures were similar. One obvious example would be wings, where you have several different strategies - feathers, insect wings, bat-like wings, etc - that have similar fun…

"Parallel" evolution is just different branches of the same evolutionary tree. The most distantly related naturally evolved lifeforms are more similar to each other than an LLM is to a human. The LLM did not evolve at all.

Re: Emotion concepts and their function in a large language model

#113

Post by mrob » Sat, Apr 04, 2026, 10:48 PM UTC

stratos123 wrote: post #92

krapp wrote: post #67

Earlier quoted context omitted.

You aren't managing the psychological state of a living thinking being. LLMs don't have "psychology." They don't actually feel emotions. They aren't actually desperate. They're trained on vast datasets of natural human language which contains the semantics of emotional interaction, so the process of matching the most statistically likely text tokens for a prompt containing emotional input tends to simulate appropriat…

Such an argument is valid for a base model , but it falls apart for anything that underwent RL training. Evolution resulted in humans that have emotions, so it's possible for something similar to arise in models during RL, e.g. as a way to manage effort when solving complex problems. It's not all that likely (even the biggest training runs probably correspond to much less optimization pressure than millenia of natura…

It's plausible that LLMs experience things during training, but during inference an LLM is equivalent to a lookup table. An LLM is a pure function mapping a list of tokens to a set of token probabilities. It needs to be connected to a sampler to make it "chat", and each token of that chat is calculated separately (barring caching, which is an implementation detail that only affects performance). There is no internal state.

Re: Emotion concepts and their function in a large language model

#114

Post by ArekDymalski » Sat, Apr 04, 2026, 10:51 PM UTC

Chance-Device wrote: post #29

Fraterkes wrote: post #24

Earlier quoted context omitted.

Do you think these llm's have subjective experiences? (by "subjective experience" I mean the thing that makes stepping on an ant worse than kicking a pebble) And if so, do you still use them? Additionaly: when do you think that subjectivity started? Was there a "there" there with gpt2?

Yes, I think they probably are conscious, though what their qualia are like might be incomprehensible to me. I don’t think that being conscious means being identical to human experience. Philosophically I don’t think there is a point where consciousness arises. I think there is a point where a system starts to be structured in such a way that it can do language and reasoning, but I don’t think these are any different…

It's not common to find just one, short post that completely changes my the worldview in a nin-trivial area. This is one of them. Thank you, that combination of mechanical interpretation + reminder that consciousness might be alien/animal but still count as consciousness was that one piece of puzzle that was missing for me. Obvious in hindsight but priceless nonetheless.

Re: Emotion concepts and their function in a large language model

#115

Post by orbital-decay » Sat, Apr 04, 2026, 10:53 PM UTC

mrob wrote: post #112

xg15 wrote: post #111

Earlier quoted context omitted.

I think a counterargument would be parallel evolution: There are various examples in nature, where a certain feature evolved independently several times, without any genetic connection - from what I understand, we believe because the evolutionary pressures were similar. One obvious example would be wings, where you have several different strategies - feathers, insect wings, bat-like wings, etc - that have similar fun…

"Parallel" evolution is just different branches of the same evolutionary tree. The most distantly related naturally evolved lifeforms are more similar to each other than an LLM is to a human. The LLM did not evolve at all.

The training process shares a lot of high-level properties with the biological evolution.

Re: Emotion concepts and their function in a large language model

#116

Post by mrob » Sat, Apr 04, 2026, 10:57 PM UTC

Chance-Device wrote: post #29

Fraterkes wrote: post #24

Earlier quoted context omitted.

Do you think these llm's have subjective experiences? (by "subjective experience" I mean the thing that makes stepping on an ant worse than kicking a pebble) And if so, do you still use them? Additionaly: when do you think that subjectivity started? Was there a "there" there with gpt2?

Yes, I think they probably are conscious, though what their qualia are like might be incomprehensible to me. I don’t think that being conscious means being identical to human experience. Philosophically I don’t think there is a point where consciousness arises. I think there is a point where a system starts to be structured in such a way that it can do language and reasoning, but I don’t think these are any different…

How can consciousness be possible without internal state? LLM inference is equivalent to repeatedly reading a giant look-up table (a pure function mapping a list of tokens to a set of token probabilities). Is the look-up table conscious merely by existing or does the act of reading it make it conscious? Does the format it's stored in make a difference?

Re: Emotion concepts and their function in a large language model

#117

Post by orbital-decay » Sat, Apr 04, 2026, 10:58 PM UTC

silentkat wrote: post #106

orbital-decay wrote: post #101

Of course they do have emotions as an internal circuit or abstraction, this is fully expected from intelligence at least at some point. But interpreting these emotions as human-like is a clear blunder. How do you tell the shoggoth likes or dislikes something, feels desperation or joy? Because it said so? How do you know these words mean the same for us? Our internal states are absolutely incompatible. We share a lot…

I like to call this Frieren's Demon. In that show, it is explained that demons evolved with no common ancestor to humans, but they speak the language. They learned the language to hunt humans. This leads to a fundamentally different understanding of words and language. Now, I don't personally believe this is an intelligence at all, but it's possible I'm wrong. What we have with these machines is a different evolution…

>it does not conceptualize a hand as a 3D object at all

Oh but it does, it's an emergent property. The biggest finding in Sora was exactly that, an internal conceptualization of the 3D space and objects. Extra fingers in older models were the result of the insufficient fidelity of this conceptualization, and also architectural artifacts in small semantically dense details.

Re: Emotion concepts and their function in a large language model

#118

Post by Kim_Bruning » Sat, Apr 04, 2026, 10:58 PM UTC

nothinkjustai wrote: post #98

salawat wrote: post #66

Earlier quoted context omitted.

>The weird part is that we're now basically managing the psychological state of our tooling, Does no one else have ethical alarm bells start ringing hardcore at statements like these? If the damn thing has a measurable psychology, mayhaps it no longer qualifies as merely a tool. Tools don't feel. Tools can't be desperate. Tools don't reward hack. Agents do. Ergo, agents aren't mere tools.

Oh no. The machine designed to output human-like text is indeed outputting human-like text. I’m half jesting; I think there is a lot of room for debate here, but I also think we shouldn’t anthropomorphize it.

Nor anthropodeny it. But really both directions are anthropocentrism in a raincoat.

Sonnet is its own thing. Which is fine.

We've known that eg. animals have emotions (functional or not) for quite a long time.

Btw: don't go looking on youtube for evidence of that. People outrageously anthropomorphizing their pets can be true at the same time.

Re: Emotion concepts and their function in a large language model

#119

Post by mrob » Sat, Apr 04, 2026, 11:01 PM UTC

orbital-decay wrote: post #115

mrob wrote: post #112

Earlier quoted context omitted.

"Parallel" evolution is just different branches of the same evolutionary tree. The most distantly related naturally evolved lifeforms are more similar to each other than an LLM is to a human. The LLM did not evolve at all.

The training process shares a lot of high-level properties with the biological evolution.

"Minimize training loss while isolated from the environment" is not at all similar to "maximize replication of genes while physically interacting with the environment". Any human-like behavior observed from LLMs is built on such fundamentally alien foundations that it can only be unreliable mimicry.

Re: Emotion concepts and their function in a large language model

#120

Post by ekidd » Sat, Apr 04, 2026, 11:02 PM UTC

tananan wrote: post #89

salawat wrote: post #66

Earlier quoted context omitted.

>The weird part is that we're now basically managing the psychological state of our tooling, Does no one else have ethical alarm bells start ringing hardcore at statements like these? If the damn thing has a measurable psychology, mayhaps it no longer qualifies as merely a tool. Tools don't feel. Tools can't be desperate. Tools don't reward hack. Agents do. Ergo, agents aren't mere tools.

When we speak of the “despair vectors”, we speak of patterns in the algorithm we can tweak that correspond to output that we recognize as despairing language . You could implement the forward pass of an LLM with pen & paper given enough people and enough time, and collate the results into the same generated text that a GPU cluster would produce. You could then ask the humans to modulate the despair vector during thei…

> I trust none of us would presume that the decentralized labor of pen & paper calculations somehow instantiated a “psychology” in the sense of a mind experiencing various levels of despair

Your argument is based on an appeal to intuition. But the scenario that you ask people to imagine is profoundly misleading in scale. Let's assume a modern frontier model, around 1 trillion parameters. Let's assume that the math is being done by an immortal monk, who can perform one weight's calculations per second.

The monk will generate the first "token", about 4 characters, in 31,688 years. In a bit over 900,000 years, the immortal monk will have generated a single Tweet.

At that point, I no longer have any intuition. The sort of math I could do by hand in a human lifetime could never "experience" anything.

But I can't rule out the possibility that 900,000 years of math might possibly become a glacial mind, expressing a brief thought across a time far greater than the human species has existed.

As the saying goes, sometimes quantity has a quality all its own.

(This is essentially the "systems response" to Searle's "Chinese room" argument. It's a old discussion.)