Emotion concepts and their function in a large language model • Hacker News Board

Re: Emotion concepts and their function in a large language model

#141

Post by xg15 » Sat, Apr 04, 2026, 11:50 PM UTC

nothinkjustai wrote: post #135

xg15 wrote: post #132

Earlier quoted context omitted.

But, well, how does it do the human-like-text-outputting exactly?

I’m guessing you aren’t just asking how an LLM works, but attempting to make the point that humans are also statistical next-token predictors or something? Humans make predictions, that doesn’t mean that’s all we do.

No, my point is that "statistical next-token predictor" is an empty phrase that doesn't really explain much. Markov chains are statistical next-token predictors as well and nevertheless, no one would confuse a markov chain with a conscious being (or deem the generated texts in any way useful for that matter).

The question is how the prediction works in detail, and those details are still being researched, as Anthropic does here, and the research can yield unexpected results.

Re: Emotion concepts and their function in a large language model

#142

Post by Kim_Bruning » Sat, Apr 04, 2026, 11:50 PM UTC

nothinkjustai wrote: post #100

stratos123 wrote: post #92

Earlier quoted context omitted.

Such an argument is valid for a base model , but it falls apart for anything that underwent RL training. Evolution resulted in humans that have emotions, so it's possible for something similar to arise in models during RL, e.g. as a way to manage effort when solving complex problems. It's not all that likely (even the biggest training runs probably correspond to much less optimization pressure than millenia of natura…

It’s a completely different substrate. LLMs don’t have agency, they don’t have a conscious, they don’t have experiences, they don’t learn over time. I’m not saying that the debate is closed, but I also think there is great danger in thinking because a machine produces human-like output, that it should be given human-like ethical considerations. Maybe in the future AI will be considered along those grounds, but…well,…

What's the empirical basis for each of your statements here? Can you enumerate? Can you provide an operational definition for each?

Re: Emotion concepts and their function in a large language model

#143

Post by lanstin » Sat, Apr 04, 2026, 11:52 PM UTC

whoiskevin wrote: post #103

nothinkjustai wrote: post #98

Earlier quoted context omitted.

Oh no. The machine designed to output human-like text is indeed outputting human-like text. I’m half jesting; I think there is a lot of room for debate here, but I also think we shouldn’t anthropomorphize it.

Completely agree here. Stop anthropomorphizing these tools. Just remove the extra language. Don't say please or thank you. Just ask for the desired outcome.

The places where solutions are discussed in a way that is best long term solution may well exist in a language subspace with politeness, calmness and thoughtfulness. Getting the model to those areas of linguistic space is useful; as is preserving my own habits of kind and thoughtful speech.

Re: Emotion concepts and their function in a large language model

#144

Post by orbital-decay » Sat, Apr 04, 2026, 11:58 PM UTC

logicprog wrote: post #107

orbital-decay wrote: post #101

Of course they do have emotions as an internal circuit or abstraction, this is fully expected from intelligence at least at some point. But interpreting these emotions as human-like is a clear blunder. How do you tell the shoggoth likes or dislikes something, feels desperation or joy? Because it said so? How do you know these words mean the same for us? Our internal states are absolutely incompatible. We share a lot…

I don't think anything you said here contradicts what they said, they take great pains throughout the blog post to explain that the model does not "expedience" these "emotions," that they're not emotions in the human sense but models of emotions (both the "expected human emotional response to a prompt" as well as what emotions another character is experiencing in part of a prompt) and functional emotions (in that the…

If you listen to Anthropic in their other works and interviews, they clearly do believe the equivalence-by-proxy between humans and LLMs to a large degree, and introduce things like model welfare (that is, caring about what the model feels). This is just another study in the series. I think they're adding these disclaimers to not sound like absolute cranks to the unprepared audience, because sometimes they really do.

Re: Emotion concepts and their function in a large language model

#145

Post by Chance-Device » Sun, Apr 05, 2026, 12:08 AM UTC

staticassertion wrote: post #93

Chance-Device wrote: post #42

Earlier quoted context omitted.

And what kind of evidence would convince you? What experiment would ever bridge this gap? You’re relying entirely on similarity between yourself and other humans. This doesn’t extend very well to anything, even animals, though more so than machines. By framing it this way have you baked in the conclusion that nothing else can be conscious on an a priori basis?

There are fields that focus on these areas and numerous ideas around what the criteria would be. One of the common understandings is that recurrent processing is likely a foundational layer for consciousness, and agents do not have this currently. I'd say that in terms of evidence I'd want to establish specific functional criteria that seem related to consciousness and then try to establish those criteria existing in…

Sorry, what are you saying? That there are people who study these things, and you’d want to see… something as evidence? Your post doesn’t actually seem to have any substantive content.

Re: Emotion concepts and their function in a large language model

#146

Post by Kim_Bruning » Sun, Apr 05, 2026, 12:09 AM UTC

logicprog wrote: post #107

orbital-decay wrote: post #101

Of course they do have emotions as an internal circuit or abstraction, this is fully expected from intelligence at least at some point. But interpreting these emotions as human-like is a clear blunder. How do you tell the shoggoth likes or dislikes something, feels desperation or joy? Because it said so? How do you know these words mean the same for us? Our internal states are absolutely incompatible. We share a lot…

I don't think anything you said here contradicts what they said, they take great pains throughout the blog post to explain that the model does not "expedience" these "emotions," that they're not emotions in the human sense but models of emotions (both the "expected human emotional response to a prompt" as well as what emotions another character is experiencing in part of a prompt) and functional emotions (in that the…

Almost! They're merely making the claim of functional emotions and outright avoiding the thorny philosophical question of whether they're "real".

[ I've actually tried exploiting functional emotions in a RAG system. The sentiment scoring and retrieval part was easy. Sentiment analysis is pretty much a settled thing I'd say, even though the mechanisms are still being studied (see the paper we're discussing.

What I'd love to be able to do is be able to extract the vector(s) they're discussing, rather than outputting as text into the context]

Re: Emotion concepts and their function in a large language model

#147

Post by Kim_Bruning » Sun, Apr 05, 2026, 12:21 AM UTC

nothinkjustai wrote: post #134

whoiskevin wrote: post #103

Earlier quoted context omitted.

Completely agree here. Stop anthropomorphizing these tools. Just remove the extra language. Don't say please or thank you. Just ask for the desired outcome.

I don’t see why you can’t use politeness. The thing is a mimic, you “treat” it badly and it mimics how a human might respond. It’s fun to play with, as long as you’re fully cognizant that IT IS NOT A HUMAN

I'd argue with you, but there's nothing strictly wrong with your statement. I'd like to point out that it's also not a cat nor a dog, nor a parrot (dead, stochastic, or otherwise). It's a Sonnet model.

Re: Emotion concepts and their function in a large language model

#148

Post by Chance-Device » Sun, Apr 05, 2026, 12:29 AM UTC

gavinray wrote: post #78

Chance-Device wrote: post #29

Earlier quoted context omitted.

Yes, I think they probably are conscious, though what their qualia are like might be incomprehensible to me. I don’t think that being conscious means being identical to human experience. Philosophically I don’t think there is a point where consciousness arises. I think there is a point where a system starts to be structured in such a way that it can do language and reasoning, but I don’t think these are any different…

I'm in the same boat with you. It's entirely too much to put in a Hacker News comment, but if I had to phrase my beliefs as precisely as possible, it would be something like: > "Phenomenal consciousness arises when a self-organizing system with survival-contingent valence runs recurrent predictive models over its own sensory and interoceptive states, and those models are grounded in a first-person causal self-tag tha…

I think your idea of consciousness is more like human/animal consciousness. Which is reasonable since that’s all we have to go off of, but I take it to mean any kind of experience, which might arise due to different types of optimisation algorithms and selective pressures.

I’m not sure I agree that everything is valence, unless I’m misunderstanding what you mean by valence. I guess it’s valence in the sense that sensory information is a specific quality with a magnitude.

I don’t think that colours, sounds and textures are somehow made out of pleasure and pain, or fear and desire. That just isn’t my subjective experience of them.

I do think that human consciousness is something like a waking dream, like how we hallucinate lots of our experiences rather than perceiving things verbatim. Perception is an active process much more than most people realise as we can see from various perceptual illusions. But I guess we’re getting more into cognition here.

Re: Emotion concepts and their function in a large language model

#149

Post by Kim_Bruning » Sun, Apr 05, 2026, 12:29 AM UTC

viralsink wrote: post #110

pbhjpbhj wrote: post #30

Earlier quoted context omitted.

Dammit, you cheated though! Why must you always do that? In your sentences it doesn't matter what your emotional state is, it makes no difference; bit like life really. Hopefully, you can see that at least my chosen sentences have an emotional aspect? An LLM could add emotional values to my previous sentences that a TTS can use for tonal variation, for example.

I can read your example in three different tonalities, of which one is the likeliest. Depending on our relationship, the interpretation could differ. The point is, the OP suggested that emotions are just a feature of language. I argue that text is one of the worst transmission channels for emotion. But I don't argue that it's not possible at all to do so, if you suggest that. That would be just silly.

Fiction writers practice really hard on this, and I'd argue that they tend to be -in the main- successful. Ish. There can still be multiple readings of a book.

Ok, I argued myself in, out, and back into that one again. It depends on the writer and the book, but a lot of writers can invoke emotion in their writing.

Fun experiment: Take a piece of creative writing (a short story); [not one of the obviously ambiguous ones, d'oh ... or do! ] and decide how it makes you feel. Ask an LLM the same question. See how far you diverge. Some LLMs give answers pretty similar to humans! If you picked an ambiguous story, see what happens if you ask for multiple readings.

Re: Emotion concepts and their function in a large language model

#150

Post by Kim_Bruning » Sun, Apr 05, 2026, 12:32 AM UTC

elcritch wrote: post #37

pbhjpbhj wrote: post #30

Earlier quoted context omitted.

Dammit, you cheated though! Why must you always do that? In your sentences it doesn't matter what your emotional state is, it makes no difference; bit like life really. Hopefully, you can see that at least my chosen sentences have an emotional aspect? An LLM could add emotional values to my previous sentences that a TTS can use for tonal variation, for example.

Makes me wonder: are there Unicode code points for tone of voice? If not could there be?

Yes, but HN apparantly filters them hard.

https://www.unicode.org/emoji/charts/full-emoji-list.html