Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A global workspace in language models

anthropic.com

Post reply on HN
Print view View original

91–100 of 218 posts

Re: A global workspace in language models

#91

Post by aesthesia »

Smaug123 wrote: post #31
minimaltom wrote: post #15

This, taken in combination with the SAE paper, the golden-gate claude paper, the feelings / introspection paper, and note in the fable system card (that they are silently nerfing responses about activation shaping), is basically confirmation to me that they have a new technique they they are using during training (along the vibe space of these mechinterp papers), and its probably some kind of representation learning…

Note that Neel Nanda replicated the results on a Qwen model.

And Neuronpedia released Jacobian lens weights for a wide range of open models: https://huggingface.co/neuronpedia/jacobian-lens

Re: A global workspace in language models

#93

Post by pants2 »

jonahrd wrote: post #86
appplication wrote: post #75

Earlier quoted context omitted.

I would need further convincing that humans do not naturally tend towards bidirectional recall. Perhaps I’m just on alert anytime I see an LLM-ism that’s met with a claim that the same or similar phenomena holds true in humans as well.

What about the fact that "What is the weird band from the early 2000s in Michigan who wore colored ties?" could be a bar trivia question (challenging enough to recall to be fun), while "Who are Tally Hall?" could not

Only because one is a short answer and the other is not

Re: A global workspace in language models

#94

Post by fc417fc802 »

hasteg wrote: post #89
fc417fc802 wrote: post #87

Earlier quoted context omitted.

A relevant question for the paper you linked is what happens if instead of repeating the input prompt you repeat a filler character the same number of times.

Can you elaborate on that more? Why just a filler character?

Because padding the output in such a manner (which keep in mind is immediately fed back as input) has been shown to increase model performance. It's one of the many reasons to question what "thinking" traces are really doing.

If you consider how the attention mechanism works then a very hand wavey intuition is that despite being entirely arbitrary additional tokens should still provide the opportunity for additional information processing.

Re: A global workspace in language models

#95

Post by winter_blue »

AlexDragusin wrote: post #85
unleaded wrote: post #55

This reminded me of some weird quirk/experiment I found with LLMs that I found while messing around, maybe someone can explain it or something. Open any AI chatbot that isn't cheating by connecting to the Internet (so disable web search). Claude, DeepSeek, Kimi, whatever. Ask them this question: "What was that weird band from michigan from the 2000s that wore coloured ties" You will probably get a wrong answer, or if…

I asked a local instance of Qwen3.5 122b: "American indie rock/pop band that formed in Ann Arbor, Michigan, in 2001 best known for their eclectic musical style, whimsical lyrics, and distinctive visual aesthetic involving matching colored ties and hats?" Got this back "The band you are describing is Tally Hall. Formed by friends at the University of Michigan, they became well-known in the mid-2000s internet era for t…

There's a shockingly unfathomable level of data compression that these local LLMs are able to remember in an encyclopedic way, so many obscure little things.

Re: A global workspace in language models

#96

Post by josh-sematic »

unleaded wrote: post #55

This reminded me of some weird quirk/experiment I found with LLMs that I found while messing around, maybe someone can explain it or something. Open any AI chatbot that isn't cheating by connecting to the Internet (so disable web search). Claude, DeepSeek, Kimi, whatever. Ask them this question: "What was that weird band from michigan from the 2000s that wore coloured ties" You will probably get a wrong answer, or if…

If I ask my kid (early elementary) what he learned last school year, I can watch him apparently genuinely trying to think of some examples and not being very successful. If I ask him something I know he learned he’ll gladly say “oh yeah!” and tell me about it.

Re: A global workspace in language models

#97

Post by nullbio »

snaking0776 wrote: post #51

This is cool but I don’t know if the comparisons to conscious awareness really make sense here. Their definition of the J-Space is basically the expectation of how much a final logits output would change as a result of a small change in a particular layer (see past work on information geometry). This seems more to me like showing there exists an abstract reasoning subspace which is generally shared across different c…

Writing it honestly would defeat the whole point of it, that being, to push the narrative that their magical token predictor is conscious. They've been trying this for years now. This video is discussing a paper they published 2 years ago by the way... It's nothing new.

Re: A global workspace in language models

#98

Post by jimbokun »

logancbrown wrote: post #28
com2kid wrote: post #21

Anyone remember that blog post from a few months back where someone was able to improve a model's math ability by just duplicating layers that were activated while solving math problems? Just literally copy/pasting them and linking them together so the model ran through the same layers again? I get the feeling a lot more research is going to come out in the area of exploring exactly what portions of a model's weights…

Source for those interested https://dnhkng.github.io/posts/rys/

Great, clear write up! Made it very easy to understand.

Re: A global workspace in language models

#99

Post by jaehong747 »

Good interpretability work, but the problem is it's all in how you interpret it. Bridge concept neurons activating even while talking about something else, this seems pretty obvious to me. Input context activating related representations is just an engineering causal structure. Call it subconscious or don't, either interpretation works. But Anthropic keeps drawing these parallels to human consciousness, and it feels intentional, like they're trying to stir up some fantasy. Kind of like comparing condensation on a camera lens to human tears. The whole point of interpretability should be clarity, not stirring up confusion. Even if some form of consciousness does exist here, it wouldn't be magic, it would be an explainable principle. Would be good if they addressed that side too.

Re: A global workspace in language models

#100

Post by ACCount37 »

ahmedfromtunis wrote: post #16

I always wondered what the model meant when it writes "I'm now considering the architecture of the service" but outputs nothing of the sorts in its CoT. Is the model really "thinking" about that stuff or is just mimicking human "manners"? And if so, where the thinking is happening if it is not in the literal chain of *thought*? I'm not sure J-Space is the answer to that question, but very interesting nevertheless.

The answer is, as is often the case, "yes".

In some cases, an LLM may truly "consider the architecture" internally, within its latent representations, and in others, it can output a similar phrase simply because it's "expected" of it.

"Where" is pretty clear. There aren't that many places within an LLM, and hidden state is the main culprit. How to read that space is another matter entirely.

Post reply on HN