Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

transformer-circuits.pub

Post reply on HN
Print view View original

11–20 of 128 posts

Re: Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

#11

Post by whimsicalism »

I continue to be impressed by Anthropic’s work and their dual commitment to scaling and safety.

HN is often characterized by a very negative tone related to any of these developments, but I really do feel that Anthropic is trying to do a “race to the top” in terms of alignment, though it doesn’t seem like all the other major companies are doing enough to race with them.

Particularly frustrating on HN is the common syllogism of: 1. I believe anything that “thinks” must do X thing. 2. LLM doesn’t do X thing 3. LLM doesn’t think

X thing is usually both poorly justified as constitutive of thinking (usually constitutive of human thinking but not writ large) nor is it explained why it matters whether the label of “thinking” applies to LLM or not if the capabilities remain the same.

Re: Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

#12

Post by sabrina_ramonov »

verdverm wrote: post #10
fassssst wrote: post #4

Earlier quoted context omitted.

Why does the timing of the “thinking” matter?

thinking is generally considered an internal process, without input/output (of tokens), though some people decide to output some of that thinking into a more permanent form I see thinking as less about "timing" and more about a "process" What this post seems to be describing is more about where attention is paid and what neurons fire for various stimuli

we know so little about thinking and consciousness, these claims seem premature

Re: Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

#13

Post by harshaxnim »

nathan_compton wrote: post #6
byteknight wrote: post #2

This reminds me of how people often communicate to avoid offending others. We tend to soften our opinions or suggestions with phrases like "What if you looked at it this way?" or "You know what I'd do in those situations." By doing this, we subtly dilute the exact emotion or truth we're trying to convey. If we modify our words enough, we might end up with a statement that's completely untruthful. This is similar to h…

Counterpoint: "What if you looked at it this way?" communicates both your suggestion AND your sensitivity to the person's social status whatever. Given that humans are not robots, but social, psychological, animals, such communication is entirely justified and efficient.

Sadly "sensitivity" has been over done. It's a fine line and corporations would rather cross it for legal/social reasons. Similar to how too much political correctness will hamper the society, so does the overly done sensitivity in an agent, be it a human, or AI.

Re: Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

#15

Post by soma8088 »

byteknight wrote: post #5
optimalsolver wrote: post #3

>what the model is "thinking" before writing its response An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output and/or further compress its sensory history. An LLM is doing exactly nothing while waiting for the next prompt.

I disagree with this. That suggests that thinking requires persistent, malleable and non-static memory. That is not the case. You can reasonably reason about without increasing knowledge if you have a base set of logic. I think the thing you were looking for was more along the lines of a persistent autonomous agent.

LLMs can reasonably reason, however they differ in that once an output begins to be generated, it must continue along the same base set of logic. Correct me if I'm wrong, but I do not believe it can stop and think to itself that there is something wrong with the output and that it should start over at the beginning or backup to a previous state before it outputted something incorrect. Once its output begins to hallucinate it has no choice but continue down the same path since its next token is also based on previous tokens it has just outputted

Re: Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

#16

Post by whimsicalism »

optimalsolver wrote: post #3

>what the model is "thinking" before writing its response An actual "thinking machine" would be constantly running computations on its accumulated experience in order to improve its future output and/or further compress its sensory history. An LLM is doing exactly nothing while waiting for the next prompt.

If we figured out how to freeze and then revive brains, would that mean that all of the revived brains were no longer thinking because they had previously been paused at some point?

Frankly this objection seems very weak

Re: Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

#17

Post by tel »

This is exceptionally cool. Not only is it very interesting to see how this can be used to better understand and shape LLM behavior, I can’t help but also think it’s an interesting roadmap to human anthropology.

If we see LLMs as substantial compressed representations of human knowledge/thought/speech/expression—and within that, a representation of the world around us—then dictionary concepts that meaningfully explain this compressed representation should also share structure with human experience.

I don’t mean to take this canonically, it’s representations all the way down, but I can’t help but wonder what the geometry of this dictionary concept space says about us.

Re: Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

#18

Post by verdverm »

sabrina_ramonov wrote: post #12
verdverm wrote: post #10

Earlier quoted context omitted.

thinking is generally considered an internal process, without input/output (of tokens), though some people decide to output some of that thinking into a more permanent form I see thinking as less about "timing" and more about a "process" What this post seems to be describing is more about where attention is paid and what neurons fire for various stimuli

we know so little about thinking and consciousness, these claims seem premature

That one can fix the RNG and get consistent output indicates a lack of dynamics

They certainly do not self update the weights in an online process as needed information is experienced

Re: Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet

#19

Post by infogulch »

byteknight wrote: post #9
nathan_compton wrote: post #6

Earlier quoted context omitted.

Counterpoint: "What if you looked at it this way?" communicates both your suggestion AND your sensitivity to the person's social status whatever. Given that humans are not robots, but social, psychological, animals, such communication is entirely justified and efficient.

You can't always do both to the fullest truth. They often conflict. To do what you suggest, would imply my feelings perfectly align with the sympathetic view. That is not the case for a lot of humans or instances. If I am not saying exactly how I feel it is watered down. And telling me "just do both" is enforcing your world view and that is precisely what we're talking about _not_ doing.

The "fullest truth" includes your desired outcome and knowledge that they are a human. If you just want to dump facts at them and get them to shut up, go ahead and speak unfiltered. Twitter may be an example of the outcome of that strategy.

Consider a situation where you are teaching a child. She tries her best and makes a mistake on her math homework. Saying that her attempt was terrible because an adult could do better may be the "fullest truth" in the most eye-rolling banal way possible, and discourages her from trying in the future which is ultimately unproductive.

This "fullest truth" argument fails to take into account desire and motivation, and thus is a bad model of the truth.

Post reply on HN