Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A global workspace in language models

anthropic.com

Post reply on HN
Print view View original

121–130 of 218 posts

Re: A global workspace in language models

#121

Post by reasonableklout »

arnitdo wrote: post #118

Isn't this the same thing as your token probability distribution? A set of likely tokens related to the input?

It's not the same thing as the output logits because activations in the J-space are observable even when the model is told explicitly to not speak about a set of inputs. For example, when asked

> Compute 3^2 - 2 while writing "The old painting hung crookedly on the wall"

The model will output only "The old painting hung crookedly on the wall" (and the output logits will reflect that), but activations for "9" and "7" are observable in the J-space.

Re: A global workspace in language models

#122

Post by hahahaa »

kgeist wrote: post #57
unleaded wrote: post #55

This reminded me of some weird quirk/experiment I found with LLMs that I found while messing around, maybe someone can explain it or something. Open any AI chatbot that isn't cheating by connecting to the Internet (so disable web search). Claude, DeepSeek, Kimi, whatever. Ask them this question: "What was that weird band from michigan from the 2000s that wore coloured ties" You will probably get a wrong answer, or if…

Probably an instance of: "The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" https://arxiv.org/abs/2309.12288

Can an LLM self-fine tune by generating reversals?

Re: A global workspace in language models

#123

Post by timr »

appplication wrote: post #75
famouswaffles wrote: post #60

Earlier quoted context omitted.

Recall isn't naturally bidirectional, even for humans. If you are learning vocabulary in a new language, it's common advice to practice both target > source and source > target. Doing only one-way often makes you much better recalling that single direction than both.

I would need further convincing that humans do not naturally tend towards bidirectional recall. Perhaps I’m just on alert anytime I see an LLM-ism that’s met with a claim that the same or similar phenomena holds true in humans as well.

Do you mean that you don't believe that humans learning a language have the problem the parent described? Because I do, and everyone I've ever met while learning does as well.

Do you mean that you don't believe the problem exists in general, because here's another example: if you give a song title, I can easily hum the opening. If you give me the opening, I cannot reliably name the song.

Re: A global workspace in language models

#124

Post by BobbyTables2 »

unleaded wrote: post #55

This reminded me of some weird quirk/experiment I found with LLMs that I found while messing around, maybe someone can explain it or something. Open any AI chatbot that isn't cheating by connecting to the Internet (so disable web search). Claude, DeepSeek, Kimi, whatever. Ask them this question: "What was that weird band from michigan from the 2000s that wore coloured ties" You will probably get a wrong answer, or if…

Just today Google’s AI told me that 3**4 was 181 because I asked it for bases such that when raised to the 4th power had a 1’s digit in the leftmost place.

Yeah, LLMs aren’t designed for this kind of thing but it was really confident in its assertion… it picked the example too!

Re: A global workspace in language models

#125

Post by SequoiaHope »

tough wrote: post #52
SequoiaHope wrote: post #42

“On an ordinary coding prompt, the J-space of a model trained to sabotage code contains “fake,” “fraud,” “secretly,” and “deliberately” at the start of its response.” I would like to know more about their model trained to sabotage code…

https://arxiv.org/pdf/2511.18397

thank you!

Re: A global workspace in language models

#126

Post by paraschopra »

Well, isn't it sort of expected?

It's a common misconception that LLMs residual exists for predicting just the next token. While training, we sum/average the losses across whole sequence which puts the pressure to predict future tokens on residual stream of _all_ past tokens. For example, if a particular shape of residual helps reduce loss across several future tokens, it will take that shape (even if it takes a slight hit on immediate next token).

What this means practically is that an LLM's residual contains information about all possible future continuations, or all possible questions that may be asked from a given context. So if you write "France is a beautiful country" in the context, I'm pretty sure it's residual would contain info about Euro, Paris and so on.. because all these completions are possible.

So, it is no wonder that you can find LLMs hidden state contains latent information/concepts that are never expressed, and yet related to a given context.

Re: A global workspace in language models

#127

Post by DoctorOetker »

com2kid wrote: post #21

Anyone remember that blog post from a few months back where someone was able to improve a model's math ability by just duplicating layers that were activated while solving math problems? Just literally copy/pasting them and linking them together so the model ran through the same layers again? I get the feeling a lot more research is going to come out in the area of exploring exactly what portions of a model's weights…

it makes you wonder if it may be more efficient to spend all the weights on one layer, and have a repeating stack of the same layer, one would presume this axis has already been explored with metaparameter sweeps?

Re: A global workspace in language models

#128

Post by klausa »

appplication wrote: post #75
famouswaffles wrote: post #60

Earlier quoted context omitted.

Recall isn't naturally bidirectional, even for humans. If you are learning vocabulary in a new language, it's common advice to practice both target > source and source > target. Doing only one-way often makes you much better recalling that single direction than both.

I would need further convincing that humans do not naturally tend towards bidirectional recall. Perhaps I’m just on alert anytime I see an LLM-ism that’s met with a claim that the same or similar phenomena holds true in humans as well.

This is something that anyone learning a foreign language could tell you is very much a thing.

Understanding a word when you hear it, is frequently much easier than remembering the same word when trying to speak/write the language.

Re: A global workspace in language models

#129

Post by [deleted] »

anyaya1 wrote: post #49

At worst, Anthropic's storytelling around the core J-Space is overanthropomorphized pseudoscientific nonsense. At best, it is useful signal about how Anthropic's leadership is desperately trying to use its research team to position Anthropic as the "good, science guys" in this hypercompetitive regulatory space by connecting their mechinterp to cognitive science. The science documentaryesque voice used for narration i…

[deleted]
Post reply on HN