Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A global workspace in language models

anthropic.com

Post reply on HN
Print view View original

151–160 of 218 posts

Re: A global workspace in language models

#151

Post by tclancy »

wavemode wrote: post #8

As someone who is not an AI researcher, the paper itself is way over my head. More interesting was the independent commentary paper they linked near the bottom: https://www-cdn.anthropic.com/files/4zrzovbb/website/cc4be24... Neel Nanda (of Google Deepmind - his part begins on page 33) discusses his opinions on the paper, and the small-scale replication he performed on an open-weight model.

Thanks for calling this out (long with others here). I am just starting in on it but had to come back to say thanks and call this out,

> We have replicated the core claims on Qwen 3.6 27B, and also share preliminary evidence of extending this work by finding abstract "interpretative meta-tokens", like Chinese characters for "what does this mean" that seem to activate and play a causal role on processing ambiguous sentences

Not sure if I am picking up what they are putting down, but if LLMs are using symbols to try to encode squishy concepts from human language into consistent, meaningful “tokens”, that sounds really interesting. In every long-term, successful use of AI, I hear echoes of The Zen of Python, “Explicit is better than implicit.” I try like hell to do it, but it’s far too easy to be lazy with AI.

Re: A global workspace in language models

#152

Post by tobr »

sigmoid10 wrote: post #139
horacemorace wrote: post #105

Earlier quoted context omitted.

If this kind of thing holds true for humans we now may understand synesthesia, perhaps we find ourselves with a large enough study pool to map out clever tricks to influence each other.

It certainly holds true for humans. The brain stores relational information in a sequential pattern that is not automatically reversible. One of the best examples is the alphabet. Everyone learns it in school, so the pattern A->B->C->... is trivial to recite for most people. Now, if I gave you a random letter to start with and asked you to to recite the remaining letters until Z, you'll probably find it is still pret…

Is the alphabet really the same though? I don't feel like I recall it one letter at a time, as individual facts linking A to B, then B to C, etc, but more as a sound or a phrase. Not unlike recalling a melody. It just seems very different from figuring out what band someone is describing.

Re: A global workspace in language models

#154

Post by tclancy »

tclancy wrote: post #151
wavemode wrote: post #8

As someone who is not an AI researcher, the paper itself is way over my head. More interesting was the independent commentary paper they linked near the bottom: https://www-cdn.anthropic.com/files/4zrzovbb/website/cc4be24... Neel Nanda (of Google Deepmind - his part begins on page 33) discusses his opinions on the paper, and the small-scale replication he performed on an open-weight model.

Thanks for calling this out (long with others here). I am just starting in on it but had to come back to say thanks and call this out, > We have replicated the core claims on Qwen 3.6 27B, and also share preliminary evidence of extending this work by finding abstract "interpretative meta-tokens", like Chinese characters for "what does this mean" that seem to activate and play a causal role on processing ambiguous sen…

Having read some more, I think we are at this stage with models:

>It is impossible to say just what I mean!

>But as if a magic lantern threw the nerves in patterns on a screen

Re: A global workspace in language models

#155

Post by tclancy »

marshray wrote: post #38
pkoiralap wrote: post #29

This is fascinating research. I feel this is a significant leap in interpretability research. Since we know J-Space exists and is bi-directional, we can train models on the same and come up with meta cognition abilities. I also fear that the big corporations might use the same to run targeted ads, capitalistic shenanigans. Which they might already be doing through system prompts.

Such an inspection capability might also be used to target ads to LLMs, which would then be more likely to mention or recommend those products and services.

The prior art here is “Have Fun, Good Luck, Don’t Die”.

Re: A global workspace in language models

#157

Post by fxwin »

unleaded wrote: post #55

This reminded me of some weird quirk/experiment I found with LLMs that I found while messing around, maybe someone can explain it or something. Open any AI chatbot that isn't cheating by connecting to the Internet (so disable web search). Claude, DeepSeek, Kimi, whatever. Ask them this question: "What was that weird band from michigan from the 2000s that wore coloured ties" You will probably get a wrong answer, or if…

I'm not sure which chatbots you used, but OpenAI's o3, o1 and 4.1 get it right first try (used in the Playground without web search or any other tools).

Re: A global workspace in language models

#158

Post by fxwin »

devanshp wrote: post #58
unleaded wrote: post #55

This reminded me of some weird quirk/experiment I found with LLMs that I found while messing around, maybe someone can explain it or something. Open any AI chatbot that isn't cheating by connecting to the Internet (so disable web search). Claude, DeepSeek, Kimi, whatever. Ask them this question: "What was that weird band from michigan from the 2000s that wore coloured ties" You will probably get a wrong answer, or if…

https://claude.ai/share/2b0f85a2-e7b8-4f62-91a0-eca61bdeabec Fable 5 on low gets the answer with web search turned off, one-shot!

fwiw o1, o3 and 4.1 also give the correct answer (without web search)

Re: A global workspace in language models

#159

Post by [deleted] »

tobr wrote: post #152
sigmoid10 wrote: post #139

Earlier quoted context omitted.

It certainly holds true for humans. The brain stores relational information in a sequential pattern that is not automatically reversible. One of the best examples is the alphabet. Everyone learns it in school, so the pattern A->B->C->... is trivial to recite for most people. Now, if I gave you a random letter to start with and asked you to to recite the remaining letters until Z, you'll probably find it is still pret…

Is the alphabet really the same though? I don't feel like I recall it one letter at a time, as individual facts linking A to B, then B to C, etc, but more as a sound or a phrase. Not unlike recalling a melody. It just seems very different from figuring out what band someone is describing.

[deleted]

Re: A global workspace in language models

#160

Post by InkCanon »

arnitdo wrote: post #118

Isn't this the same thing as your token probability distribution? A set of likely tokens related to the input?

They phrase it in a confusing way, with a lot of fluff. But to my understanding what happens is 1. At some part in the transformer, calculate the Jacobian (matrix gradients for that forward pass). Do this for 1000 prompts and average them. 2. For another pass take an "activation vector" (the input vector x to that part). Apply the matrix in step 1 to this vector. 3. Unembed the output to get a token. Presumably to make this have meaning, they use the reading of past tokens after embedding, instead of a random vector in the middle of the transformer. The high level idea appears to be you're trying to find which token, if the value were changed slightly, creates the largest downstream change.
Post reply on HN