Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-6 Astra, looped transformers, and hidden reasoning

magazine.sebastianraschka.com

Post reply on HN
Print view View original

51–60 of 169 posts

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#51

Post by XenophileJKO »

wolttam wrote: post #23

If you loop an entire transformer model on itself, that seems like by-definition hidden reasoning. If the output of the model is its reasoning trace, and you simply feed that back into the model again at inference time instead of outputting it - then it is by definition hidden (but I would expect you could pull both this trace and a further-down final output trace out)

Not really, only if you pass a residual down to the next time step.

There isn't really anything fundamentally different compared to a similar depth traditional "unrolled" model. It helps with parameter efficiency.

That doesn't mean that the model can't have "hidden" internal state, it just means it has to recompute the "hidden" part on every token inference pass without outputting it, or learn a subversive alternate meaning to words in the thought space.

This is why you see openai say that they don't want to apply direct optimization pressure on thought traces because the more the you penalize "bad thoughts" the more it could put maladaptive pressure on the reasoning tokens where they may learn "subversive meanings". It effectively damages monitoring.

Like thinking "look at" when you really mean "hack into" or even more radical coded language.

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#52

Post by shawntan »

For the research focused, there are some references in my blogpost here on what kinds of computational problems minimally require how much CoT to solve: https://blog.wtf.sg/posts/2023-02-03-the-new-xor-problem/

Notably Will Merrill's work: https://arxiv.org/abs/2310.07923

As for how universal transformers (looping transformers, but everyone has since forgotten prior work) will affect this, Will Merrill (again) has a paper here (https://arxiv.org/abs/2503.03961) that discusses exactly this.

The original universal transformers is called "universal" because if you allow for per-token looping decisions, it can theoretically be Turing complete without needing CoT (some nuance here about levels of precision used).

As for whether having little or no CoT is "unsafe": It isn't clear that the model's CoT reveal how they actually arrive at the answer. As an example, what if they provide an answer before the CoT? (https://arxiv.org/html/2603.01437v2) If this is already in question, we shouldn't be relying on the CoT for monitoring the model's reasoning.

As always there is a lot of nuance to the topic once you get your hands dirty with the details.

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#53

Post by sigbottle »

throw3954 wrote: post #8
libraryofbabel wrote: post #2

Everyone interested in LLM internals should read Sebastian. He's great. The tldr here is that the recent "The Information" article[0] reporting GPT 6 Astra was using “recurrent depth” or “looped transformers" made it sound like it was some special new scary thing ("secret technique!") that made train-of-thought monitoring harder to do. In fact, it's just the same as stacking more transformer layers, except that you r…

It’s a little more complicated than that. While looped transformers can be unrolled a fixed number of times to save on memory, if loop depth is determined dynamically between tokens, a single transformer can compute any computable function between tokens. To analogize, current transformers run a fixed-length program per step. Any program can be factored into a top-level loop with a fixed-length branching body (an int…

Nit: is it any computable function? I thought the requirements were unbounded (in principle) memory and time.

(For all intents and purposes given how high dimensional you are and using the "vibes" of computability yes I agree w/ you)

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#54

Post by redhed »

kilpikaarna wrote: post #48
redhed wrote: post #16

Earlier quoted context omitted.

I have tested it out with CAD and PCB circuits and it is a huge jump compared to Sol. I agree though when trying it with programming I don't notice a huge jump.

Anecdotally (I did try it myself, but wasn’t blown away) many seem to like it for 3D modelling. That was emphasized in the promo too. I think this kind of ”general intelligence” is what is meant to set it apart from 5.6.

Yeah my scenario was we had old paper drawings without actual CAD models. Fed those into Astra and it did it 100% perfectly. Honestly might be the easiest scenario for it, but that's also what I thought for Fable and Sol and those completely butchered it. Wish I could share pictures of those attempts but just imagine a completely mangled model that barely looks good if you squint. These were not simple models either, pretty large/complex machinery.

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#55

Post by boredatoms »

baby wrote: post #45
cainxinth wrote: post #25

Earlier quoted context omitted.

It's the same story every time OpenAI or Anthropic releases a new model. They are generous with compute for the first few days, and use maximum fidelity with uncompressed weights. Everything runs at its best to make a good first impression. But eventually they pare things back and the models perform a little worse.

You think they introduce stronger quantization after a few days?

For sure they quickly move to q8, the output quality difference to bf16 is small compared to the speed/capacity gain

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#56

Post by andriy_koval »

iJohnDoe wrote: post #9

Probably off-topic. Astra has been kind of weird. Like, I can't trust it, weird. It has an interesting tone, especially in Codex, that is off-putting. It's over zealous at times (which is why I stopped using Claude) and gets too creative when doing agentic system level stuff. Accessing files and doing things it shouldn't do. If OpenAI was chasing Claude's approach, then they are going in the wrong direction. OpenAI h…

wondering if creativity can be managed by setting reasoning level.. You pick lover reasoning for simpler tasks and high reasoning for open ended research.

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#57

Post by Alokcue »

andriy_koval wrote: post #56
iJohnDoe wrote: post #9

Probably off-topic. Astra has been kind of weird. Like, I can't trust it, weird. It has an interesting tone, especially in Codex, that is off-putting. It's over zealous at times (which is why I stopped using Claude) and gets too creative when doing agentic system level stuff. Accessing files and doing things it shouldn't do. If OpenAI was chasing Claude's approach, then they are going in the wrong direction. OpenAI h…

wondering if creativity can be managed by setting reasoning level.. You pick lover reasoning for simpler tasks and high reasoning for open ended research.

cool

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#58

Post by siva7 »

scrlk wrote: post #40
siva7 wrote: post #18

Astra was insane until Monday but something happened on tuesday, now it feels like Sol. I grieve for the lost productivity but i hope they may give us the original Astra back.

Might be related to this announcement from Tibo on Sunday: > We've made some improvements that improve usage on the long tail for power users of Astra when logged in with your ChatGPT account. > No change in quality and a pure win that on the long tail can result in up to 3-4X less usage being drawn from the subscription. https://x.com/thsottiaux/status/2096717905614524491 ( https://xcancel.com/thsottiaux/status/2096…

It seems to me the people working at OAI may believe all other humans must be a little bit behind intellectually.

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#59

Post by siva7 »

Razengan wrote: post #41
siva7 wrote: post #18

Astra was insane until Monday but something happened on tuesday, now it feels like Sol. I grieve for the lost productivity but i hope they may give us the original Astra back.

Which plan/region are you on/in?

Highest subscription tier and i believe there is only US region available being served globally

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#60

Post by simianwords »

On looped transformers:

previously, conversation might have 50k tokens spent on reasoning. the next turn takes all the previous tokens as well (if you wanna preserve prompt caching) which is not ideal. this new method skips that so you get more free context until compaction kicks in.

is this true? if so its a huge deal. why is it not spoken about? its one of the main reasons i don't use High or Max

Post reply on HN