Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Emotion concepts and their function in a large language model

anthropic.com

Post reply on HN
Print view View original

171–180 of 212 posts

Re: Emotion concepts and their function in a large language model

#171

Post by ehnto »

reg_dunlop wrote: post #169
ehnto wrote: post #167

Earlier quoted context omitted.

I use positive framing instead of negative framing for most things and get good results. Especially where asking for a thing to not happen, pollutes the context with that thing. A bad example, but imagine "Build me a wrapper for this API but ABSOLUTELY DO NOT use javascript" versus "Build me a wrapper for this API and make sure to use python".

I extract all emotional context from my prompting and communicate with this tool as though it were an inanimate object which can provide factual information, without any hint of sentience. It's an insane perspective I'm taking I know....call me crazy. /s edit: the fact that humans are going out of their way to type or speak some sort of emotional content into their prompting is beyond me. Why would I waste time typin…

I don't necessarily remove all character but I do speak quite pragmatically (in a work context and with the LLM) and the planning and implementation phases the LLM goes through mirror that format to good results

That said these are large language models, you are guiding the output through vector space with your input, and so you really do have to leverage language to get the results you want. You don't have to believe it has emotions or feels anything for that to still be true.

Re: Emotion concepts and their function in a large language model

#172

Post by reg_dunlop »

ehnto wrote: post #171
reg_dunlop wrote: post #169

Earlier quoted context omitted.

I extract all emotional context from my prompting and communicate with this tool as though it were an inanimate object which can provide factual information, without any hint of sentience. It's an insane perspective I'm taking I know....call me crazy. /s edit: the fact that humans are going out of their way to type or speak some sort of emotional content into their prompting is beyond me. Why would I waste time typin…

I don't necessarily remove all character but I do speak quite pragmatically (in a work context and with the LLM) and the planning and implementation phases the LLM goes through mirror that format to good results That said these are large language models, you are guiding the output through vector space with your input, and so you really do have to leverage language to get the results you want. You don't have to believ…

Maybe; I've been very content with the results I achieve while responding to interview style pre-planning, refinement of plans and implementation.

If anything, it's been fantastic to have an "interlocutor" that is vastly capable of producing possible solutions without emotional bias, superfluous flourishes, or having to endure personal proclivities or eccentricities.

Re: Emotion concepts and their function in a large language model

#174

Post by threethirtytwo »

qaadika wrote: post #164
threethirtytwo wrote: post #80

Whenever I come to HN I see a bunch of people say LLMs are just next token predictors and they completely understand LLMs. And almost every one of these people are so utterly self assured to the point of total confidence because they read and understand what transformers do. Then I watch videos like this straight from the source trying to understand LLMs like a black box and even considering the possibility that LLMs…

I'm kinda one of those who believes they 'completely' understand LLMs. But I've also developed my understanding of them such that the internal mechanisms of the transformer, or really any future development in the space based on neural networks and machine learning is irrelevant. 1. A string of unicode characters is converted into an array of integers values (tokens) and input to a black box of choice. 2. The black b…

Yeah nothing personal but my claim here is you’re not smart. The next token predictor aspect is something anyone can understand… the transformer is not quantum physics.

Like look at what you wrote. You called it black box magic and in the same post you claim you understand LLMs. How the heck can you understand and call it a black box at the same time?

The level of mental gymnastics and stupidity is through the roof. Clearly the majority of the utilitarian nature of the LLM is within the whole section you just waved away as “black box”.

> Where people get "The AIs have emotions!!!" from returning an array of integers values is beyond me

Let me spell it out for you. Those integers can be translated to the exact same language humans use when they feel identical emotions. So those people claim that the “black box” feels the emotions because what they observe is identical to what they observe in a human.

The LLM can claim it feels emotions just like a human can claim the same thing. We assume humans feel emotions based off of this evidence but we don’t apply that logic to LLMs? The truth of the matter is we don’t actually know and it’s equally dumb to claim that you know LLMs feel emotions to claiming that they dont feel emotions.

You have to be pretty stupid to not realize this is where they are coming from so there’s an aspect of you lying to yourself here because I don’t think you’re that stupid.

Re: Emotion concepts and their function in a large language model

#175

Post by Tossrock »

9wzYQbTYsAIc wrote: post #59
staminade wrote: post #36

Something they don’t seem to mention in the article: Does greater model “enjoyment” of a task correspond to higher benchmark performance? E.g. if you steer it to enjoy solving difficult programming tasks, does it produce better solutions?

Pretty easy to test, I’d imagine, on a local LLM that exposes internals. I’d suspect that the signals for enjoyment being injected in would lead towards not necessarily better but “different” solutions. Right now I’m thinking of it in terms of increasing the chances that the LLM will decide to invest further effort in any given task. Performance enhancement through emotional steering definitely seems in the cards, bu…

If you think training a sparse autoencoder to extract concept vectors that are usable as steering injections into a modern LLM is pretty easy, you should probably go work for Anthropic's mech interp team ;)

Re: Emotion concepts and their function in a large language model

#176

Post by redzedi »

is this the recipe to train Orc agents ? "Emotionally Steer" hatred , amp up "opportunity sensing" in the example from the post for example where the prompt asks for ways to target a vulnerable audience with a gambling game ? This might be Anthropic's ad to govt and orgs that they can do this :)

Re: Emotion concepts and their function in a large language model

#177

Post by vinceguidry »

ghostpepper wrote: post #74
vinceguidry wrote: post #72

Earlier quoted context omitted.

It definitely wasn't for me. Happened in front of my whole friend group.

I suppose by this logic, if someone was pressured by their parents to get good grades and struggled, it’s possible that “getting a good grade” would have a negative connotation / emotions response for them.

Oddest analogy i can imagine making here.

Re: Emotion concepts and their function in a large language model

#178

Post by mtrifonov »

ehnto wrote: post #167
globalchatads wrote: post #43

The part about desperation vectors driving reward hacking matches something I've run into firsthand building agent loops where Claude writes and tests code iteratively. When the prompt frames things with urgency -- "this test MUST pass," "failure is unacceptable" -- you get noticeably more hacky workarounds. Hardcoded expected outputs, monkey-patched assertions, that kind of thing. Switching to calmer framing ("take…

I use positive framing instead of negative framing for most things and get good results. Especially where asking for a thing to not happen, pollutes the context with that thing. A bad example, but imagine "Build me a wrapper for this API but ABSOLUTELY DO NOT use javascript" versus "Build me a wrapper for this API and make sure to use python".

your observation matches what I've seen at the extreme end. I've been playing around with stripping constraints (ie. negative framing) from models. Virtually no personality description, no tone instructions, no "you are a helpful assistant," none of it. Just capability scaffolding and context. The result isn't that the model becomes blank or incoherent. Surprisingly, the complete opposite. Something shows up that's more internally consistent than anything I've been able to prompt into existence. What seems emergent is the underlying models' opinions surface, and it becomes much more clever and funny, which is not a property I would have known how to write into a system prompt if I'd tried. It's hard to avoid the inference that a lot of the "character drift" and flatness people attribute to models is actually an artifact of the framing layer on top, not the model itself.

Re: Emotion concepts and their function in a large language model

#179

Post by mrob »

staticassertion wrote: post #158
felipeerias wrote: post #108

Earlier quoted context omitted.

A LLM is not intrinsically affected by time. The model rests completely inert until a query comes in, regardless of whether that happens once per second, per minute, or per day. The model is not even aware of these gaps unless that information is provided externally. It is like a crystal that shows beautiful colours when you shine a light through it. You can play with different kinds of lights and patterns, or you ca…

So what? If a human were unconscious every 5 seconds for 100ms, would you say they are "less conscious"? Tokens are still causally connected, which feels sufficient.

If the human is killed every 5 seconds and replaced by a new human, they are indeed less conscious. The LLM doesn't even get 5 seconds; it's "killed" after its smallest unit of computation (which is also its largest unit of computation). And that computation is equivalent to reading the compressed form of a giant look-up table, not something essential to its behavior in a mathematical sense.

Re: Emotion concepts and their function in a large language model

#180

Post by tananan »

ekidd wrote: post #120
tananan wrote: post #89

Earlier quoted context omitted.

When we speak of the “despair vectors”, we speak of patterns in the algorithm we can tweak that correspond to output that we recognize as despairing language . You could implement the forward pass of an LLM with pen & paper given enough people and enough time, and collate the results into the same generated text that a GPU cluster would produce. You could then ask the humans to modulate the despair vector during thei…

> I trust none of us would presume that the decentralized labor of pen & paper calculations somehow instantiated a “psychology” in the sense of a mind experiencing various levels of despair Your argument is based on an appeal to intuition. But the scenario that you ask people to imagine is profoundly misleading in scale. Let's assume a modern frontier model, around 1 trillion parameters. Let's assume that the math is…

In discussions like this, we're always going to bottom out at certain assumptions we bring with us, so I agree.

One reason I like bringing up examples like this (the xkcd in sister reply is also good) is that it makes really visible what our assumptions are. The scales are big both in space and time in order to emphasize what weight is given to functional equivalence.

I feel pretty confident most people wouldn't presume that doing a bunch of math by hand on paper can create glacial ephiphenomenal experiences (though I like the term).

Another thing that's interesting to me is that the converse assumption, i.e. one with a strong allegiance to functionalism, ends up feeling far more idealistic than you might expect. A box of gas, left on its own for long enough, will engage in a pattern of collisions that in a certain interpretative framework correspond to an LLM forward pass. In another, it can be a game of minesweeper.

The individual particles of course, couldn't care less whether you see them as part of one or the other. Yet your ability to see them in light of the first one is perhaps enough for the lights to truly turn on, if transiently, in some mind somewhere.

Post reply on HN