[dead]
GPT-6 Astra, looped transformers, and hidden reasoning
151–156 of 156 posts
Re: GPT-6 Astra, looped transformers, and hidden reasoning
#152Earlier quoted context omitted.
You need to separate memory capacity and bandwidth. Looping decreases memory capacity/FLOP but not bytes loaded/FLOP, since weights need to be loaded again for the 2nd pass. Plus (depending on the method used) capacity required for KV will be that of the equivalent unlooped model (44 blocks) and KV is typically larger than weights at long context.
Why would weights need to be « loaded again » for the 2nd pass? Weights never change at inference time no?
[flagged]
Re: GPT-6 Astra, looped transformers, and hidden reasoning
#153Earlier quoted context omitted.
Well sure, that's the possible weak point in Sebastian's article: it could be true that there's some more sophisticated stuff going on in Astra around looping, because OpenAI haven't specified their architecture. But it's always been true that, since we don't know what's in their black box, there could be arbitrary amounts of innovations inside the models that we could speculate about. So the question is, does knowin…
>Oh sure; I don't think anyone is denying that larger issue? But does it have anything to do with looping? If the model has significantly more ability to stuff away information outside visible reasoning than every other model including ones in its size class then surely it is reasonable to assume the architecture tweak that allows the model to compute more before outputing a single token is somewhat responsible for t…
> If the model has significantly more ability to stuff away information outside visible reasoning
I’m having trouble understanding why you believe the “if” part is true.
Re: GPT-6 Astra, looped transformers, and hidden reasoning
#154Astra was insane until Monday but something happened on tuesday, now it feels like Sol. I grieve for the lost productivity but i hope they may give us the original Astra back.
It's the same story every time OpenAI or Anthropic releases a new model. They are generous with compute for the first few days, and use maximum fidelity with uncompressed weights. Everything runs at its best to make a good first impression. But eventually they pare things back and the models perform a little worse.
This is a bit of an urban myth. There are trackers which keep historical performance and Sol hasn't been nerfed: https://marginlab.ai/trackers/codex/
Re: GPT-6 Astra, looped transformers, and hidden reasoning
#155For the research focused, there are some references in my blogpost here on what kinds of computational problems minimally require how much CoT to solve: https://blog.wtf.sg/posts/2023-02-03-the-new-xor-problem/ Notably Will Merrill's work: https://arxiv.org/abs/2310.07923 As for how universal transformers (looping transformers, but everyone has since forgotten prior work) will affect this, Will Merrill (again) has a…
>The original universal transformers is called "universal" because if you allow for per-token looping decisions, it can theoretically be Turing complete without needing CoT (some nuance here about levels of precision used). Looping the transformer is just as turing complete as CoT. It doesn't fundamentally grant it any new theoretical capabilities. You could just scale the model into infinity with infinite context wi…
In the limited cases (below Turing completeness) there are properties of what can be done with O(log N) depth vs O(N) CoT (regular languages), if you look at the 2nd Will Merrill paper I referenced.
Re: GPT-6 Astra, looped transformers, and hidden reasoning
#156Earlier quoted context omitted.
Run a benchmark with a large number of samples, rerun a few days later. Compare results, use statistics to see if there's a statistically significant difference.
Doing this in a way that doesn't get you noticed or fucked with is potentially going to be quite difficult. If they have a "hey we're being benchmarked" mode, which is not hard to imagine, avoiding tripping it is going to be annoying and difficult to prove.
You would want to use your own benchmark, certainly, not something publicly known. But outside of that I don't think "this is a benchmark" is such an easy category to determine -- a benchmark should be similar to a typical scenario anyway.