[dead]
GPT-6 Astra, looped transformers, and hidden reasoning
151–160 of 169 posts
Re: GPT-6 Astra, looped transformers, and hidden reasoning
#152Earlier quoted context omitted.
You need to separate memory capacity and bandwidth. Looping decreases memory capacity/FLOP but not bytes loaded/FLOP, since weights need to be loaded again for the 2nd pass. Plus (depending on the method used) capacity required for KV will be that of the equivalent unlooped model (44 blocks) and KV is typically larger than weights at long context.
Why would weights need to be « loaded again » for the 2nd pass? Weights never change at inference time no?
[flagged]
Re: GPT-6 Astra, looped transformers, and hidden reasoning
#153Earlier quoted context omitted.
Well sure, that's the possible weak point in Sebastian's article: it could be true that there's some more sophisticated stuff going on in Astra around looping, because OpenAI haven't specified their architecture. But it's always been true that, since we don't know what's in their black box, there could be arbitrary amounts of innovations inside the models that we could speculate about. So the question is, does knowin…
>Oh sure; I don't think anyone is denying that larger issue? But does it have anything to do with looping? If the model has significantly more ability to stuff away information outside visible reasoning than every other model including ones in its size class then surely it is reasonable to assume the architecture tweak that allows the model to compute more before outputing a single token is somewhat responsible for t…
> If the model has significantly more ability to stuff away information outside visible reasoning
I’m having trouble understanding why you believe the “if” part is true.
Re: GPT-6 Astra, looped transformers, and hidden reasoning
#154Astra was insane until Monday but something happened on tuesday, now it feels like Sol. I grieve for the lost productivity but i hope they may give us the original Astra back.
It's the same story every time OpenAI or Anthropic releases a new model. They are generous with compute for the first few days, and use maximum fidelity with uncompressed weights. Everything runs at its best to make a good first impression. But eventually they pare things back and the models perform a little worse.
This is a bit of an urban myth. There are trackers which keep historical performance and Sol hasn't been nerfed: https://marginlab.ai/trackers/codex/
Re: GPT-6 Astra, looped transformers, and hidden reasoning
#155For the research focused, there are some references in my blogpost here on what kinds of computational problems minimally require how much CoT to solve: https://blog.wtf.sg/posts/2023-02-03-the-new-xor-problem/ Notably Will Merrill's work: https://arxiv.org/abs/2310.07923 As for how universal transformers (looping transformers, but everyone has since forgotten prior work) will affect this, Will Merrill (again) has a…
>The original universal transformers is called "universal" because if you allow for per-token looping decisions, it can theoretically be Turing complete without needing CoT (some nuance here about levels of precision used). Looping the transformer is just as turing complete as CoT. It doesn't fundamentally grant it any new theoretical capabilities. You could just scale the model into infinity with infinite context wi…
In the limited cases (below Turing completeness) there are properties of what can be done with O(log N) depth vs O(N) CoT (regular languages), if you look at the 2nd Will Merrill paper I referenced.
Re: GPT-6 Astra, looped transformers, and hidden reasoning
#156Earlier quoted context omitted.
Run a benchmark with a large number of samples, rerun a few days later. Compare results, use statistics to see if there's a statistically significant difference.
Doing this in a way that doesn't get you noticed or fucked with is potentially going to be quite difficult. If they have a "hey we're being benchmarked" mode, which is not hard to imagine, avoiding tripping it is going to be annoying and difficult to prove.
You would want to use your own benchmark, certainly, not something publicly known. But outside of that I don't think "this is a benchmark" is such an easy category to determine -- a benchmark should be similar to a typical scenario anyway.
Re: GPT-6 Astra, looped transformers, and hidden reasoning
#157Re: GPT-6 Astra, looped transformers, and hidden reasoning
#158Earlier quoted context omitted.
>Oh sure; I don't think anyone is denying that larger issue? But does it have anything to do with looping? If the model has significantly more ability to stuff away information outside visible reasoning than every other model including ones in its size class then surely it is reasonable to assume the architecture tweak that allows the model to compute more before outputing a single token is somewhat responsible for t…
> If the model has significantly more ability to stuff away information outside visible reasoning I’m having trouble understanding why you believe the “if” part is true.
Because it is.
From OpenAI
9.2.1 CoT Controllability
We find that GPT-6 Astra’s CoT controllability is substantially higher than that of GPT-5.6 Sol and GPT-5.5 Thinking (Figure 28)....For example, among CoTs between 750 and 1,250 tokens long, GPT-6 Astra successfully controls 60.9%, compared with 16.1% for GPT-5.6 Sol and 1.7% for GPT-5.5 Thinking. This increase in controllability is consistent across the three datasets (Figure 29) and across the eight CoT instruction types (Figure 30). Qualitatively, GPT-6 Astra is now capable of generating very long CoTs satisfying complex constraints, e.g., alternating between lowercase and uppercase letters (Table 9) and pretending to reason about a different question (Table 10).[1]
9.3 External Evaluation for Monitorability - UK AISI
To assess monitorability, UK AISI evaluated Astra using four non-agentic evaluations:
No-CoT math time horizon: Astra can solve significantly more difficult math problems in a single forward pass than past models. UK AISI measured Astra’s time horizon at 30.9 minutes compared to 3.6 minutes for GPT 5.6 Sol (Figure 1). [2]
[1]https://deploymentsafety.openai.com/gpt-6-astra/cot-controll...
[2]https://deploymentsafety.openai.com/gpt-6-astra/external-eva...
Outside OpenAI
Astra has 8.6x better odds of doing a reasoning task without CoT than the next best model (Fable 5.1), and can do 7.2 serial arithmetic steps in a forward pass vs 4.1 for the next best model (Gemini 3.8 Flash/Fable 5.1) [3]
[3]https://www.lesswrong.com/posts/eRmzz8J8Qkzqvzrgg/astra-can-...
[4]https://www.lesswrong.com/posts/ntKx9YHWCwxSeGbRB/estimating...
[5]https://www.lesswrong.com/posts/FsCkkoGsNmPzFKRhg/gpt-6-astr...
Re: GPT-6 Astra, looped transformers, and hidden reasoning
#159The MSPAINT computer use demo made my jaw drop. I guess it's not too different from the SVG pelicans, in terms of what it's doing, but it's still amazing to see it working in real-time like that.
This one's even better. Using Canva https://x.com/iam_zachi/status/2095992132620136677
what the actual fuck
Re: GPT-6 Astra, looped transformers, and hidden reasoning
#160Earlier quoted context omitted.
Doing this in a way that doesn't get you noticed or fucked with is potentially going to be quite difficult. If they have a "hey we're being benchmarked" mode, which is not hard to imagine, avoiding tripping it is going to be annoying and difficult to prove.
You would want to use your own benchmark, certainly, not something publicly known. But outside of that I don't think "this is a benchmark" is such an easy category to determine -- a benchmark should be similar to a typical scenario anyway.
> "this is a benchmark" is such an easy category to determine
I mean, it's not _that_ hard to determine most likely, and/or it's hard to be sure you didn't get found out by llm-assisted analysis on your traffic. It's not going to be a one-shot request and response it's going to be a whole bunch of them in an artificial way, by nature.
And then anything you found is single-use only if you're paranoid because even if they find out later, they have your benchmark now (because you sent it to them to use it even if you don't publish it).