A global workspace in language models • Hacker News Board

Re: A global workspace in language models

#11

Post by lucrbvi » Mon, Jul 06, 2026, 7:42 PM UTC

bilsbie wrote: post #6

I’m confused where in the weights the jspace is.

Anthropic theorize that middle layers in an LLM is a "J-Space" used to "think" about the future answer or about abstract concepts.

Their method is used to identify which tokens can appears in which layers of the model.

Re: A global workspace in language models

#12

Post by charcircuit » Mon, Jul 06, 2026, 7:51 PM UTC

meatmanek wrote: post #3

It would be really cool if they could expose this information to customers somehow. Imagine: - having a log of the most prominent J-space tokens during your customer support chatbot's interactions with a user, so you can have more introspection into why a particular outcome happened - being able to detect certain thoughts associated with undesirable behavior (hallucinations, overstepping authority, lying, etc.) and t…

Anthropic aren't even willing to expose the CoT of their models. You will have to rely on them to build those sorts of things into dedicated signals.

Re: A global workspace in language models

#13

Post by viralsink » Mon, Jul 06, 2026, 7:54 PM UTC

blauditore wrote: post #10

esafak wrote: post #2

Without using the term, they are using an information geometric approach.

But J-Space is much catchier. This is not a scientific paper, it's a promotional essay.

First button on the page is a link to the scientific paper. It's called "Read the paper". You'll find an explanation for the term in there.

Re: A global workspace in language models

#14

Post by botanrice » Mon, Jul 06, 2026, 7:58 PM UTC

[flagged]

Re: A global workspace in language models

#15

Post by minimaltom » Mon, Jul 06, 2026, 7:59 PM UTC

This, taken in combination with the SAE paper, the golden-gate claude paper, the feelings / introspection paper, and note in the fable system card (that they are silently nerfing responses about activation shaping), is basically confirmation to me that they have a new technique they they are using during training (along the vibe space of these mechinterp papers), and its probably some kind of representation learning akin to the core ideas of JEPA.

(Nb: not an expert / in the labs, just opining)

Re: A global workspace in language models

#16

Post by ahmedfromtunis » Mon, Jul 06, 2026, 7:59 PM UTC

I always wondered what the model meant when it writes "I'm now considering the architecture of the service" but outputs nothing of the sorts in its CoT.

Is the model really "thinking" about that stuff or is just mimicking human "manners"? And if so, where the thinking is happening if it is not in the literal chain of *thought*?

I'm not sure J-Space is the answer to that question, but very interesting nevertheless.

Re: A global workspace in language models

#17

Post by shevy-java » Mon, Jul 06, 2026, 8:02 PM UTC

As long as language models are liars, such as documented here recently:

https://distrowatch.com/weekly.php?issue=20260706#freebsd

We should really stop giving these liar models any further credibility.

Re: A global workspace in language models

#18

Post by llmslave » Mon, Jul 06, 2026, 8:02 PM UTC

I cannot wait for the machine god

Re: A global workspace in language models

#19

Post by wongarsu » Mon, Jul 06, 2026, 8:06 PM UTC

ahmedfromtunis wrote: post #16

I always wondered what the model meant when it writes "I'm now considering the architecture of the service" but outputs nothing of the sorts in its CoT. Is the model really "thinking" about that stuff or is just mimicking human "manners"? And if so, where the thinking is happening if it is not in the literal chain of *thought*? I'm not sure J-Space is the answer to that question, but very interesting nevertheless.

Almost none of the hosted models give you their unredacted CoT. Claude certainly doesn't, what you get are fragments and summaries from it.

There are various justifications on this, but it's mostly to make distillation and fine tuning off their model outputs a bit harder for their competitors

Re: A global workspace in language models

#20

Post by baq » Mon, Jul 06, 2026, 8:09 PM UTC

ahmedfromtunis wrote: post #16

I always wondered what the model meant when it writes "I'm now considering the architecture of the service" but outputs nothing of the sorts in its CoT. Is the model really "thinking" about that stuff or is just mimicking human "manners"? And if so, where the thinking is happening if it is not in the literal chain of *thought*? I'm not sure J-Space is the answer to that question, but very interesting nevertheless.

> I'm now considering the architecture of the service

What you see here is a summary of thinking tokens written by some other smaller model (e.g. old sonnet). The actual thinking sometimes (rarely) leaks and is not easy to parse.