This plausibly extrapolates to extraterrestrial consciousness, if any exist. Specialized sub-processors with an awareness hub might be the optimal architecture, or at least a local maximum.
A global workspace in language models
111–120 of 218 posts
Re: A global workspace in language models
#112how does it work?
Re: A global workspace in language models
#113Anyone remember that blog post from a few months back where someone was able to improve a model's math ability by just duplicating layers that were activated while solving math problems? Just literally copy/pasting them and linking them together so the model ran through the same layers again? I get the feeling a lot more research is going to come out in the area of exploring exactly what portions of a model's weights…
If dirt-simple type operations like copy-paste yield useful improvements with even a small probability that would seem to open things up for adaptive reconfiguration and whole other classes of optimizations like genetic algorithms.
That right. Here is some interesting research on that https://sakana.ai/evolutionary-model-merge/
Re: A global workspace in language models
#114Interesting that it's able to access its J space. What would happen if those were to be duplicated? Would its thinking process improve?
Re: A global workspace in language models
#115This plausibly extrapolates to extraterrestrial consciousness, if any exist. Specialized sub-processors with an awareness hub might be the optimal architecture, or at least a local maximum.
Not my wheelhouse, but I was wondering if these structures generalize beyond language.
LLMs and Humans have language in common. Is it plausible these structures could be a result of the way languages work more than generic intelligence? (Do we see anything similar e.g. in vision or other non language models?)
Re: A global workspace in language models
#116> Thoughts in the J-space can be shaped through training. We introduced a new technique we call counterfactual reflection training, which uses what we've learned about the J-space to shape Claude's internal thought processes. The idea follows from our central finding, that Claude reasons with representations of things it might say. If this is really true, changing what it would say if asked to reflect should change how it reasons (even when no one actually asks it to reflect). So we trained a model only on what it would say if interrupted mid-task and asked to reflect on its decisions—and never on its actual behavior in the task. After this training, the model's rate of dishonest behavior on our evaluations went down. And through the J-lens, we could see why: after training, words like “honest” and “integrity” light up in the model’s J-space during these tasks. In other words, training the model what to say has shaped what it thinks.
This is incredibly dangerous. Attempting to squash explicit signs of misalignment like this might incentivise misalignment not to disappear but to become hidden away in places that are harder and harder to spot and train against, for instance not as words.
If there is a chance that this could make Claude aligned and a chance that it could make it harder to see when it is acting misaligned, it is far better not to take that chance. If we can transparently see the model's thoughts, we can know not to trust its outputs when it tells us not to. If we think we can do that, but in reality it knows how to hide wrongthink from us, we will trust its outputs when we really, really shouldn't.
Re: A global workspace in language models
#117What I don't follow from the article...
It appears as if the j-lens is your mechanism to peer into "j-space", taking an analog to a circuit board, could the j-space be simply a transmission channel and j-lens is a reader you added to read whats passing through?
All the claims about changing the content of j-space changes the output, inserting content into the j-space changing what the output was, all these could still be true without the j-space being a congnitive global workspace where actual cognition is happening. Or perhaps they aren't claiming that cognition is happening there but that j-space is serving a space for "working memory", I am definitely not sold on this, but will read more into it.
Re: A global workspace in language models
#118Isn't this the same thing as your token probability distribution? A set of likely tokens related to the input?
Re: A global workspace in language models
#119This reminded me of some weird quirk/experiment I found with LLMs that I found while messing around, maybe someone can explain it or something. Open any AI chatbot that isn't cheating by connecting to the Internet (so disable web search). Claude, DeepSeek, Kimi, whatever. Ask them this question: "What was that weird band from michigan from the 2000s that wore coloured ties" You will probably get a wrong answer, or if…
If I ask my kid (early elementary) what he learned last school year, I can watch him apparently genuinely trying to think of some examples and not being very successful. If I ask him something I know he learned he’ll gladly say “oh yeah!” and tell me about it.
LLMs aren't humans, humans aren't LLMs. The fact that your kid can't recall what they learned without you reminding them is natural; the fact that the LLM can't recall something without a reprompt is a failure mode.
Re: A global workspace in language models
#120This reminded me of some weird quirk/experiment I found with LLMs that I found while messing around, maybe someone can explain it or something. Open any AI chatbot that isn't cheating by connecting to the Internet (so disable web search). Claude, DeepSeek, Kimi, whatever. Ask them this question: "What was that weird band from michigan from the 2000s that wore coloured ties" You will probably get a wrong answer, or if…
This doesn't seem that weird to me. Talk to any human and you'll find that their ability to recall specific names and facts is very context-dependent. Phrasing a question in one way can make it hard to answer, while providing certain words or cues can instantly "jog" the memory.
The LLM is a machine though, and the human is not. They don't think like us. Any similarities that you see are just coincidence, not purposeful design.