Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-6 Astra, looped transformers, and hidden reasoning

magazine.sebastianraschka.com

Post reply on HN
Print view View original

71–80 of 151 posts

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#71

Post by andai »

Anecdatum but I experienced looped cognition on a peculiar combination of substances.

I was able to treat thoughts as solid objects and manipulate them iteratively. (Ordinarily they're more like "glimpses" or "flashes" that fade rapidly. So I guess it would be like the mental equivalent of tracers.)

I was able to stack thoughts on top of each other, like planks. (I can do something similar or the narrowly but the planks are not nearly as wide!)

I didn't do any tests unfortunately but subjectively my cognition was greatly enhanced. (Spent a few years catching up with the insights I had that evening.)

Might be unrelated, but the part about "looped transformers" made me wonder if there's a similar "stepwise" increment going on here.

Edit: Okay, 6.8-18% is slightly less dramatic than what I was referring to.

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#72

Post by dooglius »

cainxinth wrote: post #25
siva7 wrote: post #18

Astra was insane until Monday but something happened on tuesday, now it feels like Sol. I grieve for the lost productivity but i hope they may give us the original Astra back.

It's the same story every time OpenAI or Anthropic releases a new model. They are generous with compute for the first few days, and use maximum fidelity with uncompressed weights. Everything runs at its best to make a good first impression. But eventually they pare things back and the models perform a little worse.

Do you have hard evidence of this assertion?

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#73

Post by AnimalMuppet »

enraged_camel wrote: post #24
iJohnDoe wrote: post #9

Probably off-topic. Astra has been kind of weird. Like, I can't trust it, weird. It has an interesting tone, especially in Codex, that is off-putting. It's over zealous at times (which is why I stopped using Claude) and gets too creative when doing agentic system level stuff. Accessing files and doing things it shouldn't do. If OpenAI was chasing Claude's approach, then they are going in the wrong direction. OpenAI h…

>> It's over zealous at times (which is why I stopped using Claude) and gets too creative when doing agentic system level stuff. Accessing files and doing things it shouldn't do. I gave Astra a pretty straightforward bug ticket yesterday. The bug involved an edge case that could sometimes result in an invalid value getting stored in a user profile field. Pretty harmless, no crash or anything, just annoying. Based on…

You have to watch it like a hawk so it doesn't do something to production, on its own, without a specific request? Wow. Then I could never trust it to not be doing something to some other system that it shouldn't, so I'd have to audit every network request.

If enraged_camel had been doing something else involving the production database at the wrong time, they might have accepted the 1Password prompt.

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#74

Post by famouswaffles »

libraryofbabel wrote: post #7
famouswaffles wrote: post #3

Earlier quoted context omitted.

>made it sound like it was some special new scary thing that made train-of-thought monitoring harder to do. It's not a "scary new thing" but ultimately no-one knows exactly how OpenAI have implemented looping. You might not be aware/remember but MoE transformers perennially underperfomed their dense counterparts until GPT-4. Similarly, making reinforcement learning really work with transformers wasn't figured out unt…

Well sure, that's the possible weak point in Sebastian's article: it could be true that there's some more sophisticated stuff going on in Astra around looping, because OpenAI haven't specified their architecture. But it's always been true that, since we don't know what's in their black box, there could be arbitrary amounts of innovations inside the models that we could speculate about. So the question is, does knowin…

>Oh sure; I don't think anyone is denying that larger issue? But does it have anything to do with looping?

If the model has significantly more ability to stuff away information outside visible reasoning than every other model including ones in its size class then surely it is reasonable to assume the architecture tweak that allows the model to compute more before outputing a single token is somewhat responsible for this change ?

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#75

Post by atomflunder3000 »

I only used Astra while coding a bit so I can't comment on anything else but I have been really disappointed by it.

It seems to overengineer really bad and it is also very slow due to it "thinking" too much I feel like.

One example is that I asked it to implement a new functionality inside an existing App of mine and if I had written it myself it would have been like a ~50 line diff. Astra took like 10 minutes to write ~400 lines, most of them useless and also in pretty bad style, barely readable code.

Maybe I am bad with prompting but I didn't have these issues before, not even with 5.6 Sol on max reasoning.

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#76

Post by famouswaffles »

WhitneyLand wrote: post #43
wolttam wrote: post #23

If you loop an entire transformer model on itself, that seems like by-definition hidden reasoning. If the output of the model is its reasoning trace, and you simply feed that back into the model again at inference time instead of outputting it - then it is by definition hidden (but I would expect you could pull both this trace and a further-down final output trace out)

No. It’s not at all by definition hidden reasoning. Looping transformers uses additional calculations (repeating layers) to generate a token. Reasoning (in this context) is test time generation of multiple tokens that allow a model to have a scratch pad to refine its thoughts, chain of thought reasoning in other words. Doing the former in no way means that you have to hide the latter. Raschka is right in this post, T…

It can lead to hidden reasoning, if the looping allows it to stuff enough information outside visible CoT. Open AI demostrates such an ability by asking it to solve problems while thinking about something else entirely. All the other models are unable to do this except Astra. It doesn't have to be a substitute for CoT to cause monitorability issues.

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#77

Post by andai »

> So, the whole idea here is that we increase the effective depth from 22 to 44 block applications without adding another set of transformer weights.

From what I gathered, LLM inference is bottlenecked on memory, right? Which implies there's "spare" compute we haven't been using? Does reusing the weights like this allow us to utilize it? (Do more math per unit of memory?)

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#78

Post by simlevesque »

dooglius wrote: post #72
cainxinth wrote: post #25

Earlier quoted context omitted.

It's the same story every time OpenAI or Anthropic releases a new model. They are generous with compute for the first few days, and use maximum fidelity with uncompressed weights. Everything runs at its best to make a good first impression. But eventually they pare things back and the models perform a little worse.

Do you have hard evidence of this assertion?

We can't have hard evidence. It's a SaaS and they own the code and the machine it runs on.

So it may be a widespread hallucination. But there's no evidence of that either.

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#79

Post by bradly »

dooglius wrote: post #72
cainxinth wrote: post #25

Earlier quoted context omitted.

It's the same story every time OpenAI or Anthropic releases a new model. They are generous with compute for the first few days, and use maximum fidelity with uncompressed weights. Everything runs at its best to make a good first impression. But eventually they pare things back and the models perform a little worse.

Do you have hard evidence of this assertion?

There is a toot from an Open AI person a couple days ago saying they are "pulling all the levers" because of capacity issues. I have no idea what the heck the person is talking about, but I'm guessing there are consequence for those levers.

    > "Demand for Astra is really unprecedented. We're pulling all the levers possible to sustain the demand, but I've not seen anything like it until now and we went through very steep growth before. Priority will always be to keep excellent service for existing users, but we might have to pause new Pro subscriptions for a bit if this continues."

Re: GPT-6 Astra, looped transformers, and hidden reasoning

#80

Post by Vetch »

cainxinth wrote: post #25
siva7 wrote: post #18

Astra was insane until Monday but something happened on tuesday, now it feels like Sol. I grieve for the lost productivity but i hope they may give us the original Astra back.

It's the same story every time OpenAI or Anthropic releases a new model. They are generous with compute for the first few days, and use maximum fidelity with uncompressed weights. Everything runs at its best to make a good first impression. But eventually they pare things back and the models perform a little worse.

The most charitable explanation I can think of for this is something like regression to the mean. When a model is first released, there'll be a subset of users who, just by chance, sample the highest quality band of the distribution that answers their query. Some of them will rush over to social media and post about how amazing a model is. Over time, those users' mental model of responses will converge but they'll perceive the model's return to typical performance as a downgrade.

This guess/explanation predicts that most users won't match what the initial social media hype claims, doesn't discount user experience as simple habituation nor does it assume companies are lying when they say there have been no changes to the model itself (quantization included).

I also think there's an aspect where initial testing is more forgiving because the more persnickety polish bits can be ignored and tests are likely to have similar structure to things that can be trained for. Meanwhile, actual specific work items are a broader unusual distribution with more stringent acceptance criteria.

Personally, I can detect a separation between Sol and Astra (but not as large as that between Opus and Fable). While they can solve most of the same problems, Astra takes less time, is less frustrating to talk to, is cleaner, notices more, spins wheels less and requires less corrections.

Post reply on HN