Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Emotion concepts and their function in a large language model

anthropic.com

Post reply on HN
Print view View original

131–140 of 212 posts

Re: Emotion concepts and their function in a large language model

#131

Post by Chance-Device »

mrob wrote: post #116
Chance-Device wrote: post #29

Earlier quoted context omitted.

Yes, I think they probably are conscious, though what their qualia are like might be incomprehensible to me. I don’t think that being conscious means being identical to human experience. Philosophically I don’t think there is a point where consciousness arises. I think there is a point where a system starts to be structured in such a way that it can do language and reasoning, but I don’t think these are any different…

How can consciousness be possible without internal state? LLM inference is equivalent to repeatedly reading a giant look-up table (a pure function mapping a list of tokens to a set of token probabilities). Is the look-up table conscious merely by existing or does the act of reading it make it conscious? Does the format it's stored in make a difference?

What state is lacking? There is a result which requires computation to be output. The model is the state. The computation must be performed for each input to produce a given output. What are you even objecting to?

Re: Emotion concepts and their function in a large language model

#132

Post by xg15 »

nothinkjustai wrote: post #98
salawat wrote: post #66

Earlier quoted context omitted.

>The weird part is that we're now basically managing the psychological state of our tooling, Does no one else have ethical alarm bells start ringing hardcore at statements like these? If the damn thing has a measurable psychology, mayhaps it no longer qualifies as merely a tool. Tools don't feel. Tools can't be desperate. Tools don't reward hack. Agents do. Ergo, agents aren't mere tools.

Oh no. The machine designed to output human-like text is indeed outputting human-like text. I’m half jesting; I think there is a lot of room for debate here, but I also think we shouldn’t anthropomorphize it.

But, well, how does it do the human-like-text-outputting exactly?

Re: Emotion concepts and their function in a large language model

#133

Post by twodave »

whoiskevin wrote: post #103
nothinkjustai wrote: post #98

Earlier quoted context omitted.

Oh no. The machine designed to output human-like text is indeed outputting human-like text. I’m half jesting; I think there is a lot of room for debate here, but I also think we shouldn’t anthropomorphize it.

Completely agree here. Stop anthropomorphizing these tools. Just remove the extra language. Don't say please or thank you. Just ask for the desired outcome.

Indeed. It reminds me of Lewis’ That Hideous Strength in a way. If we take the severed head post-brain-death and pump it with blood and oxygen and feed it impulses so that the mouth moves to form the words we tell it, is the person living again? No, it’s just a head, speaking the words it’s been given.

Re: Emotion concepts and their function in a large language model

#134

Post by nothinkjustai »

whoiskevin wrote: post #103
nothinkjustai wrote: post #98

Earlier quoted context omitted.

Oh no. The machine designed to output human-like text is indeed outputting human-like text. I’m half jesting; I think there is a lot of room for debate here, but I also think we shouldn’t anthropomorphize it.

Completely agree here. Stop anthropomorphizing these tools. Just remove the extra language. Don't say please or thank you. Just ask for the desired outcome.

I don’t see why you can’t use politeness. The thing is a mimic, you “treat” it badly and it mimics how a human might respond.

It’s fun to play with, as long as you’re fully cognizant that IT IS NOT A HUMAN

Re: Emotion concepts and their function in a large language model

#135

Post by nothinkjustai »

xg15 wrote: post #132
nothinkjustai wrote: post #98

Earlier quoted context omitted.

Oh no. The machine designed to output human-like text is indeed outputting human-like text. I’m half jesting; I think there is a lot of room for debate here, but I also think we shouldn’t anthropomorphize it.

But, well, how does it do the human-like-text-outputting exactly?

I’m guessing you aren’t just asking how an LLM works, but attempting to make the point that humans are also statistical next-token predictors or something?

Humans make predictions, that doesn’t mean that’s all we do.

Re: Emotion concepts and their function in a large language model

#136

Post by nothinkjustai »

Kim_Bruning wrote: post #118
nothinkjustai wrote: post #98

Earlier quoted context omitted.

Oh no. The machine designed to output human-like text is indeed outputting human-like text. I’m half jesting; I think there is a lot of room for debate here, but I also think we shouldn’t anthropomorphize it.

Nor anthropodeny it. But really both directions are anthropocentrism in a raincoat. Sonnet is its own thing. Which is fine. We've known that eg. animals have emotions (functional or not) for quite a long time. Btw: don't go looking on youtube for evidence of that. People outrageously anthropomorphizing their pets can be true at the same time.

What is there to anthropodeny?

Re: Emotion concepts and their function in a large language model

#137

Post by NooneAtAll3 »

orbital-decay wrote: post #101

Of course they do have emotions as an internal circuit or abstraction, this is fully expected from intelligence at least at some point. But interpreting these emotions as human-like is a clear blunder. How do you tell the shoggoth likes or dislikes something, feels desperation or joy? Because it said so? How do you know these words mean the same for us? Our internal states are absolutely incompatible. We share a lot…

> interpreting these emotions as human-like is a clear blunder. How do you tell the shoggoth likes or dislikes something, feels desperation or joy? Because it said so? How do you know these words mean the same for us?

I think you took it backwards

those vectors are exactly what it says - it affects the output and we can measure it

and it's exactly what it means for us because that's what it's measured against

and the main problem isn't "is its emotion same as ours", but "does it apply our emotion as emotion"

Re: Emotion concepts and their function in a large language model

#138

Post by nothinkjustai »

Kim_Bruning wrote: post #123
whoiskevin wrote: post #103

Earlier quoted context omitted.

Completely agree here. Stop anthropomorphizing these tools. Just remove the extra language. Don't say please or thank you. Just ask for the desired outcome.

Okay great, that's EASILY operatinalizable. Set up -say- 100 replications of the same question sequence (say to build a program) against some cheap model like qwen. One half of the set can be with please and thank you, and the other half without. You can vibe code it even. I'd be curious to see your results!

You can even boost its effectiveness by roleplaying with it. I’m not joking. Fully based on vibes, I haven’t done any testing. But it’s part of prompting imo.

IMO these things are like a reflection. Present what you want reflected back.

Re: Emotion concepts and their function in a large language model

#139

Post by Erem »

astrange wrote: post #99
Erem wrote: post #88

Earlier quoted context omitted.

Why is it more monstrous to alter weights post-training than to do so as part of curating the training corpus? After all we already control these activation patterns through the system prompt by which we summon a character out of the model. This just provides more fine grain control

It would be more moral to give the LLM a tool call that lets it apply steering to itself. Similar to how you'd prefer to give a person antipsychotics at home rather than put them in a mental hospital.

Why is it in the moral axis at all? I imagine identifying and shaping the influence of unwanted emotion vectors would happen as data selection in pretraining or natural feedback loops during the rl phase, same as we shape unwanted output for current models in order to make them practical and helpful

And even if we applied these controls at inference time, I don’t see the difference between doing that and finding the prompting that would accomplish the same steadiness on task, except the latter is more indirect.

Re: Emotion concepts and their function in a large language model

#140

Post by buu700 »

ekidd wrote: post #120
tananan wrote: post #89

Earlier quoted context omitted.

When we speak of the “despair vectors”, we speak of patterns in the algorithm we can tweak that correspond to output that we recognize as despairing language . You could implement the forward pass of an LLM with pen & paper given enough people and enough time, and collate the results into the same generated text that a GPU cluster would produce. You could then ask the humans to modulate the despair vector during thei…

> I trust none of us would presume that the decentralized labor of pen & paper calculations somehow instantiated a “psychology” in the sense of a mind experiencing various levels of despair Your argument is based on an appeal to intuition. But the scenario that you ask people to imagine is profoundly misleading in scale. Let's assume a modern frontier model, around 1 trillion parameters. Let's assume that the math is…

I don't personally believe LLMs are sentient, but I've always enjoyed this thought experiment: https://xkcd.com/505. I have a signed copy framed on my wall.
Post reply on HN