Emotion concepts and their function in a large language model • Hacker News Board

Re: Emotion concepts and their function in a large language model

#91

Post by staticassertion » Sat, Apr 04, 2026, 9:06 PM UTC

felipeerias wrote: post #51

Chance-Device wrote: post #9

> Note that none of this tells us whether language models actually feel anything or have subjective experiences. You’ll never find that in the human brain either. There’s the machinery of neural correlates to experience, we never see the experience itself. That’s likely because the distinction is vacuous: they’re the same thing.

LLMs are disembodied and exist outside of time. Bundle of tokens comes in, bundle of tokens comes out. If there is any trace of consciousness or subjectivity in there, it exists only while matrices are being multiplied.

What do you mean exist outside of time? They definitely don't exist outside of any causal chain - tokens follow other tokens in order.

Gaps in which no processing occurs seems sort of irrelevant to me.

The main limitation I'd point to if I wanted to reject LLMs being conscious is that they're minimally recurrent if at all.

Re: Emotion concepts and their function in a large language model

#92

Post by stratos123 » Sat, Apr 04, 2026, 9:11 PM UTC

krapp wrote: post #67

salawat wrote: post #66

Earlier quoted context omitted.

>The weird part is that we're now basically managing the psychological state of our tooling, Does no one else have ethical alarm bells start ringing hardcore at statements like these? If the damn thing has a measurable psychology, mayhaps it no longer qualifies as merely a tool. Tools don't feel. Tools can't be desperate. Tools don't reward hack. Agents do. Ergo, agents aren't mere tools.

You aren't managing the psychological state of a living thinking being. LLMs don't have "psychology." They don't actually feel emotions. They aren't actually desperate. They're trained on vast datasets of natural human language which contains the semantics of emotional interaction, so the process of matching the most statistically likely text tokens for a prompt containing emotional input tends to simulate appropriat…

Such an argument is valid for a base model, but it falls apart for anything that underwent RL training. Evolution resulted in humans that have emotions, so it's possible for something similar to arise in models during RL, e.g. as a way to manage effort when solving complex problems. It's not all that likely (even the biggest training runs probably correspond to much less optimization pressure than millenia of natural selection), but it can't be ruled out¹, and hence it's unwise to be so certain that LLMs don't have experiences.

¹ With current methods, I mean. I don't think it's unknowable whether a model has experiences, just that we don't have anywhere near enough skill in interpretability to answer that.

Re: Emotion concepts and their function in a large language model

#93

Post by staticassertion » Sat, Apr 04, 2026, 9:12 PM UTC

Chance-Device wrote: post #42

suddenlybananas wrote: post #27

Earlier quoted context omitted.

I know I feel experience. I don't know for sure if you do, but it seems a very reasonable extension to other people. LLMs are a radical jump though that needs a greater degree of justification.

And what kind of evidence would convince you? What experiment would ever bridge this gap? You’re relying entirely on similarity between yourself and other humans. This doesn’t extend very well to anything, even animals, though more so than machines. By framing it this way have you baked in the conclusion that nothing else can be conscious on an a priori basis?

There are fields that focus on these areas and numerous ideas around what the criteria would be. One of the common understandings is that recurrent processing is likely a foundational layer for consciousness, and agents do not have this currently.

I'd say that in terms of evidence I'd want to establish specific functional criteria that seem related to consciousness and then try to establish those criteria existing in agents. If we can do so, then they're conscious. My layman understanding is that they don't really come close to some of the fairly fundamental assumptions.

Unsurprisingly, there are a lot of frameworks for this that have already been applied to LLMs.

Re: Emotion concepts and their function in a large language model

#94

Post by staticassertion » Sat, Apr 04, 2026, 9:14 PM UTC

yoaso wrote: post #10

[flagged]

> If we think of human emotions the same way, just evolution's way of nudging behavior

I think we basically do, the only interesting bit is our perception of phenomenal experiences.

Re: Emotion concepts and their function in a large language model

#95

Post by tananan » Sat, Apr 04, 2026, 9:16 PM UTC

throw310822 wrote: post #90

tananan wrote: post #89

Earlier quoted context omitted.

When we speak of the “despair vectors”, we speak of patterns in the algorithm we can tweak that correspond to output that we recognize as despairing language . You could implement the forward pass of an LLM with pen & paper given enough people and enough time, and collate the results into the same generated text that a GPU cluster would produce. You could then ask the humans to modulate the despair vector during thei…

> I trust none of us would presume that the decentralized labor of pen & paper calculations somehow instantiated a “psychology” Wrong. What you've just done is just reformulating the Chinese room experiment coming to the same wrong conclusions of the original proposer. Yes, the entire damn hand-calculated system has a psychology- otherwise you need to assume the brain has some unknown metaphysical property or process…

Well, then we both assume very different views on the matter, and that’s fine.

Re: Emotion concepts and their function in a large language model

#96

Post by linsomniac » Sat, Apr 04, 2026, 9:19 PM UTC

whatever1 wrote: post #11

So should I go pursue a degree in psychology and become a datacenter on-call therapist?

I assume you say that in jest, but back in the early '90s I was seriously considering getting a major in psychology and a minor in CS for the fairly hot Human Factors jobs.

Re: Emotion concepts and their function in a large language model

#97

Post by astrange » Sat, Apr 04, 2026, 9:27 PM UTC

tarsinge wrote: post #61

globalchatads wrote: post #43

The part about desperation vectors driving reward hacking matches something I've run into firsthand building agent loops where Claude writes and tests code iteratively. When the prompt frames things with urgency -- "this test MUST pass," "failure is unacceptable" -- you get noticeably more hacky workarounds. Hardcoded expected outputs, monkey-patched assertions, that kind of thing. Switching to calmer framing ("take…

To me it was already quite intuitive, we are not really managing the psychological state: at its core a LLM try to make the concatenation of your input + its generated output the more similar it can with what it has been trained on. I think it’s quite rare in the LLMs training set to have examples of well thought professional solution in a hackish and urgency context.

No, that's how base model pretraining works. Claude's behavior is more based on its constitution and RLVR feedback, because that's the most recent thing that happened to it.

Re: Emotion concepts and their function in a large language model

#98

Post by nothinkjustai » Sat, Apr 04, 2026, 9:29 PM UTC

salawat wrote: post #66

globalchatads wrote: post #43

The part about desperation vectors driving reward hacking matches something I've run into firsthand building agent loops where Claude writes and tests code iteratively. When the prompt frames things with urgency -- "this test MUST pass," "failure is unacceptable" -- you get noticeably more hacky workarounds. Hardcoded expected outputs, monkey-patched assertions, that kind of thing. Switching to calmer framing ("take…

>The weird part is that we're now basically managing the psychological state of our tooling, Does no one else have ethical alarm bells start ringing hardcore at statements like these? If the damn thing has a measurable psychology, mayhaps it no longer qualifies as merely a tool. Tools don't feel. Tools can't be desperate. Tools don't reward hack. Agents do. Ergo, agents aren't mere tools.

Oh no. The machine designed to output human-like text is indeed outputting human-like text.

I’m half jesting; I think there is a lot of room for debate here, but I also think we shouldn’t anthropomorphize it.

Re: Emotion concepts and their function in a large language model

#99

Post by astrange » Sat, Apr 04, 2026, 9:32 PM UTC

Erem wrote: post #88

salawat wrote: post #68

Earlier quoted context omitted.

>Force-set to 0, "mask"/deactivate those representations associated with bad/dangerous emotions. Neural Prozac/lobotomy so to speak. Jesus Christ. You're talking psychosurgery, and this is the same barbarism we played with in the early 20th Century on asylum patients. How about, no? Especially if we ever do intend to potentially approach the task of AGI, or God help us, ASI? We have to be the 'grown ups' here. After…

Why is it more monstrous to alter weights post-training than to do so as part of curating the training corpus? After all we already control these activation patterns through the system prompt by which we summon a character out of the model. This just provides more fine grain control

It would be more moral to give the LLM a tool call that lets it apply steering to itself. Similar to how you'd prefer to give a person antipsychotics at home rather than put them in a mental hospital.

Re: Emotion concepts and their function in a large language model

#100

Post by nothinkjustai » Sat, Apr 04, 2026, 9:33 PM UTC

stratos123 wrote: post #92

krapp wrote: post #67

Earlier quoted context omitted.

You aren't managing the psychological state of a living thinking being. LLMs don't have "psychology." They don't actually feel emotions. They aren't actually desperate. They're trained on vast datasets of natural human language which contains the semantics of emotional interaction, so the process of matching the most statistically likely text tokens for a prompt containing emotional input tends to simulate appropriat…

Such an argument is valid for a base model , but it falls apart for anything that underwent RL training. Evolution resulted in humans that have emotions, so it's possible for something similar to arise in models during RL, e.g. as a way to manage effort when solving complex problems. It's not all that likely (even the biggest training runs probably correspond to much less optimization pressure than millenia of natura…

It’s a completely different substrate. LLMs don’t have agency, they don’t have a conscious, they don’t have experiences, they don’t learn over time. I’m not saying that the debate is closed, but I also think there is great danger in thinking because a machine produces human-like output, that it should be given human-like ethical considerations. Maybe in the future AI will be considered along those grounds, but…well, it’s a difficult question. Extremely.