[dead]
GPT-6 Astra, looped transformers, and hidden reasoning
151–152 of 152 posts
Re: GPT-6 Astra, looped transformers, and hidden reasoning
#152Earlier quoted context omitted.
You need to separate memory capacity and bandwidth. Looping decreases memory capacity/FLOP but not bytes loaded/FLOP, since weights need to be loaded again for the 2nd pass. Plus (depending on the method used) capacity required for KV will be that of the equivalent unlooped model (44 blocks) and KV is typically larger than weights at long context.
Why would weights need to be « loaded again » for the 2nd pass? Weights never change at inference time no?
[flagged]