Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A global workspace in language models

anthropic.com

Post reply on HN
Print view View original

181–190 of 218 posts

Re: A global workspace in language models

#181

Post by mike_hearn »

dnhkng wrote: post #177
mike_hearn wrote: post #170

Earlier quoted context omitted.

Nah, it's a cool blog post especially as it was real AI research done at home (albeit with a ridiculously expensive PC), but Anthropic and other labs have been investigating this kind of thing for years. Even the original transformer architecture makes this clear. It had an explicit "encoder" phase and then a "decoder" phase. Modern LLMs collapse the two together, or are sometimes described rather confusingly as bein…

Author here: Yeah, the encoder and decoder stuff is explicit, but the internal structure in generated during training. I don't think the big labs were doing this back when I did the research; no one was back in '24. I just didn't get round to publishing for years, because I have a day job. By the way, it still works! I tested it earlier this year on Qen3.6 and you still see improvements, so either a) no one actually…

I think you're right that the idea of looping layers is unique to you, congrats and thanks for writing those great blog posts (I read them for the first time a few days ago!). But the idea that the thinking is happening in an abstract space via neural circuits in the middle layers I feel was one that I was reading about in 2024 at least, as Anthropic have been doing this kind of research for a long time. Maybe I'm misremembering though!

My impression from reading the literature is that there are a gazillion interesting ideas and findings published that nobody is picking up in production models. The big labs are researcher constrained, there just aren't enough hours in the day to keep up with the literature and integrate all the interesting ideas found there. So it's not surprising that your trick still works. It'd be even less surprising to discover nobody at these labs has read your blogs, or they have but never found time to experiment with them. Or, they tried, but there is no set of loops that improves some metrics without harming others - I would expect neural circuits to be misaligned across the middle layers so looping layers for one task would put a fault line in circuits for other tasks.

Then they have to trade off the extra GPU capacity needed to do the extra layers, and so on.

Re: A global workspace in language models

#182

Post by vinceguidry »

tobr wrote: post #152
sigmoid10 wrote: post #139

Earlier quoted context omitted.

It certainly holds true for humans. The brain stores relational information in a sequential pattern that is not automatically reversible. One of the best examples is the alphabet. Everyone learns it in school, so the pattern A->B->C->... is trivial to recite for most people. Now, if I gave you a random letter to start with and asked you to to recite the remaining letters until Z, you'll probably find it is still pret…

Is the alphabet really the same though? I don't feel like I recall it one letter at a time, as individual facts linking A to B, then B to C, etc, but more as a sound or a phrase. Not unlike recalling a melody. It just seems very different from figuring out what band someone is describing.

Ask someone to name all fifty states, see how far they get. Then start naming places and ask if they're a US state or not. One of these tasks is far more likely to get a 100% hit rate.

Re: A global workspace in language models

#183

Post by stanfordkid »

sigmoid10 wrote: post #139
horacemorace wrote: post #105

Earlier quoted context omitted.

If this kind of thing holds true for humans we now may understand synesthesia, perhaps we find ourselves with a large enough study pool to map out clever tricks to influence each other.

It certainly holds true for humans. The brain stores relational information in a sequential pattern that is not automatically reversible. One of the best examples is the alphabet. Everyone learns it in school, so the pattern A->B->C->... is trivial to recite for most people. Now, if I gave you a random letter to start with and asked you to to recite the remaining letters until Z, you'll probably find it is still pret…

I think what you're saying might be a stretch. I strongly believe the brain holds some information in a sequential pattern -- not necessarily all of it or even the majority of it. There's a book called Moonwalking with Einstein that digs into this precise fact -- you can remember sequences much much better if you associate a visual image with each item in the sequence. Sequential association covers a lot of human knowledge but certainly not the spatial aspects. I think it's a big reason why I think LLM's might not be as smart as we think they are. The complex spatial representations are only encoded implicitly via projection on to textual descriptions.

Re: A global workspace in language models

#184

Post by tobr »

dryarzeg wrote: post #173
tobr wrote: post #152

Earlier quoted context omitted.

Is the alphabet really the same though? I don't feel like I recall it one letter at a time, as individual facts linking A to B, then B to C, etc, but more as a sound or a phrase. Not unlike recalling a melody. It just seems very different from figuring out what band someone is describing.

Not that I'm too sure about how this applies (or whether it applies at all) to other people as well, but for me personally it's easier to recall information about the band if I'm being told the band's name instead of being told "well, they are known for coloured ties". So, there is certainly some kind of effect described in action. Now, about alphabet: again, I think it's only me, but when I try to recall it backward…

After writing the comment I thought about it some more, and realized when asking myself the question ”what letter comes after R?”, I didn’t immediately know the answer, but I heard ”MNOPQRS” in my head which gave me the answer. So I feel like I know it because I know the rhythm and sound of saying those letters in sequence.

Re: A global workspace in language models

#185

Post by tedd4u »

mike_hearn wrote: post #181
dnhkng wrote: post #177

Earlier quoted context omitted.

Author here: Yeah, the encoder and decoder stuff is explicit, but the internal structure in generated during training. I don't think the big labs were doing this back when I did the research; no one was back in '24. I just didn't get round to publishing for years, because I have a day job. By the way, it still works! I tested it earlier this year on Qen3.6 and you still see improvements, so either a) no one actually…

I think you're right that the idea of looping layers is unique to you, congrats and thanks for writing those great blog posts (I read them for the first time a few days ago!). But the idea that the thinking is happening in an abstract space via neural circuits in the middle layers I feel was one that I was reading about in 2024 at least, as Anthropic have been doing this kind of research for a long time. Maybe I'm mi…

The Ouro looping results are interesting [1] and they are focused more on the improved reasoning from looping middle layers rather than the parameter efficiency aspect. They train 1.4 and 2.6B parameter models with 7T tokens. The training includes learning how many times to loop on any given token (there’s an early exit module). My guess as to why (as far as we know) looping is not in frontier models yet is that, at frontier training run scale, it’s probably going to require a lot of trial and error and at-scale research. While currently they already probably have a list of dozens or hundreds of of promising ideas that don’t complicate things as much. In the other hand, Ouro’s looping technique shows ability to compete well with models with 3x parameters which seems attention-getting to me. If there’s another 3x to be had down that path. It’s order of magnitude opportunity. Btw there is a great related work section in the paper.

[1] https://ouro-llm.github.io/

Re: A global workspace in language models

#186

Post by naasking »

unleaded wrote: post #55

This reminded me of some weird quirk/experiment I found with LLMs that I found while messing around, maybe someone can explain it or something. Open any AI chatbot that isn't cheating by connecting to the Internet (so disable web search). Claude, DeepSeek, Kimi, whatever. Ask them this question: "What was that weird band from michigan from the 2000s that wore coloured ties" You will probably get a wrong answer, or if…

Not sure why this is weird when you really think about it. You're saying that a multiparameter constraint satisfaction problem is harder to solve, even when there's only a single solution (one point in the total space that satisfies all constraints), than just being given that solution/point. Well of course it is!

Re: A global workspace in language models

#187

Post by sigmoid10 »

stanfordkid wrote: post #183
sigmoid10 wrote: post #139

Earlier quoted context omitted.

It certainly holds true for humans. The brain stores relational information in a sequential pattern that is not automatically reversible. One of the best examples is the alphabet. Everyone learns it in school, so the pattern A->B->C->... is trivial to recite for most people. Now, if I gave you a random letter to start with and asked you to to recite the remaining letters until Z, you'll probably find it is still pret…

I think what you're saying might be a stretch. I strongly believe the brain holds some information in a sequential pattern -- not necessarily all of it or even the majority of it. There's a book called Moonwalking with Einstein that digs into this precise fact -- you can remember sequences much much better if you associate a visual image with each item in the sequence. Sequential association covers a lot of human kno…

But it does cover the state described in the top comment. AB during training, regardless if you are a human neural network or an artificial one. Also, you need to define "spatial" better. This is about logic after all and not geometry. Or topology? It's unclear which context you refer to. It's certainly not the topic of this thread.

Re: A global workspace in language models

#188

Post by tedd4u »

kgeist wrote: post #69

Judging by the examples, if I understand it correctly, J-space supports higher-order logical / multihop transformations, but it is limited in size because of the limited network depth (max number of layers). When we emulate "reasoning," we basically extend J-space and allow the higher-order transformations to continue for longer, toward a more logical conclusion. It sounds like instead of generating reasoning tokens…

See also Ouro [1]. Good citations in the “related work” section.

[1] https://ouro-llm.github.io/

Re: A global workspace in language models

#189

Post by tedd4u »

POBIX wrote: post #116

> Thoughts in the J-space can be shaped through training. We introduced a new technique we call counterfactual reflection training, which uses what we've learned about the J-space to shape Claude's internal thought processes. The idea follows from our central finding, that Claude reasons with representations of things it might say. If this is really true, changing what it would say if asked to reflect should change h…

Right, will the devious unaligned thoughts squish over to a “K-space” we (the trainers) are not aware of?

Re: A global workspace in language models

#190

Post by icandrive »

redwood wrote: post #172
icandrive wrote: post #171

> What color is the planet fourth from the sun? Are they trying to show internal consistency even when the produced answer is wrong?

I dont understand what your comment is referring to

In the second figure with the title "Functional roles of the global workspace" the third example asks this question and the model answers incorrectly.
Post reply on HN