Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A global workspace in language models

anthropic.com

Post reply on HN
Print view View original

141–150 of 218 posts

Re: A global workspace in language models

#141

Post by jll29 »

paraschopra wrote: post #126

Well, isn't it sort of expected? It's a common misconception that LLMs residual exists for predicting just the next token. While training, we sum/average the losses across whole sequence which puts the pressure to predict future tokens on residual stream of _all_ past tokens. For example, if a particular shape of residual helps reduce loss across several future tokens, it will take that shape (even if it takes a slig…

"France is a beautiful country" may also still continued by "...in the heart of Europe".

Re: A global workspace in language models

#142

Post by arnitdo »

reasonableklout wrote: post #121
arnitdo wrote: post #118

Isn't this the same thing as your token probability distribution? A set of likely tokens related to the input?

It's not the same thing as the output logits because activations in the J-space are observable even when the model is told explicitly to not speak about a set of inputs. For example, when asked > Compute 3^2 - 2 while writing "The old painting hung crookedly on the wall" The model will output only "The old painting hung crookedly on the wall" (and the output logits will reflect that), but activations for "9" and "7"…

A model does not speak, it only guesses the next possible token. Activations for 9 and 7 will most likely be observed after "The ", but increasingly become improbable after "old"

Re: A global workspace in language models

#143

Post by fasterik »

nozzlegear wrote: post #120
fasterik wrote: post #74

Earlier quoted context omitted.

This doesn't seem that weird to me. Talk to any human and you'll find that their ability to recall specific names and facts is very context-dependent. Phrasing a question in one way can make it hard to answer, while providing certain words or cues can instantly "jog" the memory.

The LLM is a machine though, and the human is not. They don't think like us. Any similarities that you see are just coincidence, not purposeful design.

I still don't see why it's surprising, though. Humans and machines are subject to the same laws of physics. While we wouldn't expect LLMs to think exactly like humans, we should expect some degree of convergent evolution. There are always tradeoffs between storage capacity, accuracy, latency, relevance, fluency, etc.

Expecting LLMs to be magically perfect information retrieval machines is never going to be realistic.

Re: A global workspace in language models

#144

Post by jll29 »

NotGMan wrote: post #30

>> None of this tells us whether Claude is conscious in the way people are, or whether it feels anything at all My problem with the entire "Is AI conscious" debate is that we don't even know what exactly consciousness in humans is. You need to understand something in order to compare it to something else. Otherwise you are just comparing different definitions and second order derived phenomena.

In short, it's the "mind's 'I'" - we think not as response to external stimuli (only) such as prompts, but we have an inner "I" that asks questions on its own initiative. There are people like Douglas R. Hofstadter, who believe consciousness is not linked to human hardware (the brain), but that it is an epiphenomenon that emerges as a result of sufficient complexity of the underlying system: https://en.wikipedia.org/wiki/The_Mind%27s_I

I believe that while underlying high complexity is certainly logically necessary for consciousness, but it is not logically sufficient, and I am undecided (slightly "pro" intuitively) on the question of separability of consciousness from its hardware.

Will a LLM ask an original question on day? I doubt it.

Note that AI models do not have to be conscious to be useful (or to take away millions of jobs)!

Re: A global workspace in language models

#145

Post by abrbhat »

unleaded wrote: post #55

This reminded me of some weird quirk/experiment I found with LLMs that I found while messing around, maybe someone can explain it or something. Open any AI chatbot that isn't cheating by connecting to the Internet (so disable web search). Claude, DeepSeek, Kimi, whatever. Ask them this question: "What was that weird band from michigan from the 2000s that wore coloured ties" You will probably get a wrong answer, or if…

P(i|j) != P(j|i)

Re: A global workspace in language models

#146

Post by andy12_ »

paraschopra wrote: post #126

Well, isn't it sort of expected? It's a common misconception that LLMs residual exists for predicting just the next token. While training, we sum/average the losses across whole sequence which puts the pressure to predict future tokens on residual stream of _all_ past tokens. For example, if a particular shape of residual helps reduce loss across several future tokens, it will take that shape (even if it takes a slig…

I think what's unexpected is that it seems that some cases of model errors are truly caused by the model being misaligned? In the "Catching a model fabricating data" example I would have thought that it was just the model being stupid and not understanding the intent of the question, but as per its J-Space, it seems the model is "aware" in some sense that it's manipulating/faking data?

There is also now a deeper question. When a model is misaligned deception-related tokens seem to appear in its J-Space. But this happens only when the model is "aware" in some sense that it is misaligned. What happens if they do not? Is it possible to create a model so misaligned that itself is not aware that is is misaligned? How would you detect such thing?

Re: A global workspace in language models

#147

Post by unleaded »

lukan wrote: post #140
unleaded wrote: post #80

Earlier quoted context omitted.

What does it say to the second question? I've found Claude is one of the worst models with regards to pop culture knowledge like this, even compared to the Chinese open ones. Just curious, not really relevant to the initial post but I don't pay for it so I only have access to Sonnet. https://claude.ai/share/5e7e09b2-a75a-4024-b261-9a1a4e063a8b this is mostly hilariously wrong. wrong tie colors, they did not replace t…

"I've found Claude is one of the worst models with regards to pop culture knowledge like this, " Is that bad? I want to use models for coding and reasoning capabilities, not pop trivia knowledge they can get with web search.

For most cases probably not. It's just something I like testing new models on sometimes, the pelican riding a bicycle benchmark probably isn't that useful either.

Re: A global workspace in language models

#148

Post by IsTom »

winter_blue wrote: post #95
AlexDragusin wrote: post #85

Earlier quoted context omitted.

I asked a local instance of Qwen3.5 122b: "American indie rock/pop band that formed in Ann Arbor, Michigan, in 2001 best known for their eclectic musical style, whimsical lyrics, and distinctive visual aesthetic involving matching colored ties and hats?" Got this back "The band you are describing is Tally Hall. Formed by friends at the University of Michigan, they became well-known in the mid-2000s internet era for t…

There's a shockingly unfathomable level of data compression that these local LLMs are able to remember in an encyclopedic way, so many obscure little things.

Is it? Compressed (text) Wikipedia is in ballpark of dozens of GBs.

Re: A global workspace in language models

#149

Post by TeMPOraL »

ACCount37 wrote: post #73
optimalsolver wrote: post #64

Earlier quoted context omitted.

So the model was updated in the 37 minutes since OP posted his comment?

Just a funny observation. Every time someone proclaims "LLMs can't do X", a bigger, badder LLM that can in fact do X shows up shortly thereafter. Clearly, Fable 5 didn't even have the decency to wait until the next model refresh cycle to show up. It was already sitting there waiting. Either the capability gains in bigger, badder models are actually unrelated to "gotchas" being discovered, or LLMs are already acquirin…

Or just AI denialists like to say "LLMs can't do X" even though they can and have been doing it for the past few months or more. They only get called out once the current SOTA LLMs get so good at it, that any rando can trivially and reliably falsify the claim on the spot with whatever SOTA LLM surface they have handy.

Which I suspect is what happened here, given the trail of smaller / local models that successfully answer the question, too.

That said, "curse of reversability" is real, as much for LLMs as it is for people.

Re: A global workspace in language models

#150

Post by lukan »

unleaded wrote: post #147
lukan wrote: post #140

Earlier quoted context omitted.

"I've found Claude is one of the worst models with regards to pop culture knowledge like this, " Is that bad? I want to use models for coding and reasoning capabilities, not pop trivia knowledge they can get with web search.

For most cases probably not. It's just something I like testing new models on sometimes, the pelican riding a bicycle benchmark probably isn't that useful either.

But that is just testing for encoded knowledge, the pelican riding requires some reasoning capabilities, but lost its surely usefullness a while ago.
Post reply on HN