Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI reduces Codex Model Context Size from 372k to 272k

github.com

Post reply on HN
Print view View original

81–90 of 194 posts

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#81

Post by jubilanti »

damsta wrote: post #20

Not a fan of their context compaction and I feel like 1M-token context should minimum today. Each day I see how GPT 5.5 and 5.6 struggle a bit after each compaction before they get to the full speed, sometimes focusing too much on some older steering message that made it into the compacted context.

The fact there is no way to disable auto-compaction like every other coding harness makes me scream. It fires randomly at 10-20%, so really you now only have guaranteed context of 272k * 80%. So many times I've had it work on a large codebase, it is almost done with a task, it needs maybe a 2000 token response, but it drops below 20%, it churns and churns and churns and then I see the dreaded ``Context compacted`` and scream. You can't go back in the history to the state of the conversation before the compaction. So then it has to investigate the codebase again gets to 20% and then auto-compacts. So it then has to read more, and then auto-compacts again, and then I'm out of tokens.

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#82

Post by planckscnst »

jeremyjh wrote: post #65
tekacs wrote: post #33

Earlier quoted context omitted.

I've implemented a similar approach – although I'm surprised not to see mention of cache prefix busting in there!

Yes the idea is cool but this could really hammer usage, especially just leaving it up to the agent to decide when to do it. I'm not surprised though, considering the github account is named "Vibecodelicious" and became active in December. Poking around in the repo the whole implementation is an unsupervised LLM fever-dream.

It depends on what you mean by "unsupervised" - I've been strictly working through the agent, through specs the entire time, but it's been very supervised, I just leave the mistakes in-place and have it work from there.

However at this point it can completely maintain itself. When a new version of Claude or OpenCode is released, it updates itself to work on the latest version. It can also add new implementations for harnesses pretty reliably. It's actually pretty fun to watch it at this point. "Make this work on Hermes agent and message me when you're done" and an hour later or so, I can go play with it in Hermes.

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#83

Post by KronisLV »

On long sessions even with workflows and sub-agents I typically hit around 500-750k tokens in context (say 4-16 hours of work in a session, across multiple 5-hour limits) with Claude Code.

I was going to explore paying for Codex since OpenAI seems to be a bit more generous with rate limits, but I'm now not sure - for the stuff I do limited context size would be a dealbreaker - basically lots and lots of documentation and guideline references, code review loops, documentation references and web searches, multi-repo exploration, lots of tool calls and so on.

I might just upgrade to Anthropic's more expensive Max subscription or something to get more subsidized tokens. Even with plan files and the plan mode, it's like a slot machine after compacting the context, sometimes steps or other details just evaporate in thin air, the less of that I need to do the better. It's not that it can't work, it just doesn't work reliably enough not to be annoying.

Luckily DeepSeek V4 Pro, GLM 5.2 and Kimi K3 don't seem to have those limits either - though DS is around Sonnet, GLM 5.2 feels a bit above Sonnet and only K3 only really is in the proper Opus ballpark that's good enough for me to work undisturbed, even if it seems to be slower.

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#84

Post by ethin »

tekacs wrote: post #26

I know a lot of people like to say that compaction makes this moot, but the level of detail you lose across compaction is wildly too much for most things that I do, unfortunately. Perhaps if your plans don't have as much detail, or if you're not, for example, having a discussion with a lot of nitty-gritty then it's fine? The lack of long context is the main reason that I still end up using Anthropic. The worst is whe…

I suffered this same problem with Anthropic, ironically. Mainly with reverse engineering things. That would usually end up requiring a ton of context it had to remember, and then compaction would kick in and it would just forget half of it (that half being really important too), and even switching to Claude Code and having it document everything wouldn't really fix the problem. It was honestly quite sad because I've usually gotten better results with it, but it falls a part the moment you try doing something like that that's really complicated and requires a ton of different attempts to get it right.

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#85

Post by p1esk »

jubilanti wrote: post #79
tekacs wrote: post #26

I know a lot of people like to say that compaction makes this moot, but the level of detail you lose across compaction is wildly too much for most things that I do, unfortunately. Perhaps if your plans don't have as much detail, or if you're not, for example, having a discussion with a lot of nitty-gritty then it's fine? The lack of long context is the main reason that I still end up using Anthropic. The worst is whe…

The fact there is no way to disable auto-compaction and no way to go back in the conversation history to before a compact makes codex a no-go for me on any codebase > 5kloc. Even worse it seems to fire randomly at 10-20% of context left. So really you have 80% of 272k as real usable context. Compaction kills my sessions, it hallucinates and is worse than starting fresh. I've had enough times screaming at my computer…

[flagged]

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#86

Post by throwatdem12311 »

I primarily use Opus for my day to day and I /clear often. 1M context sound great but it quickly degrades once you start hitting the 50% mark. I usually just /clear once I hit between 30 and 40 percent context and get much better results from the model. Compaction doesn’t really help much in my experience, starting fresh and having the model front load its context from scratch works much better IME. I have multiple skills with a pile of markdown docs organized by feature to assist in that first load by telling it where to find relevant information for the task. Works great.

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#87

Post by conradkay »

brandoncarl wrote: post #55

With purely quadratic attention, the cost of the token at 372K is 87% more than the token at 272K. There are attention mechanisms that help to mitigate this, but you can clearly see in the chart that Tibo posted that the attention mechanisms they are using are still scaling quadratically. This becomes an optimization problem for token t: min f[cost, quality loss] = compaction cost[t] + compaction quality loss[t] + to…

It's not quadratic attention, you get that curve from the input tokens going up linearly, since the graph is measuring cumulative cost at each token count. Basically for y=5 it's 5+4+3+2+1, or f(x) = x(x+1)/2

https://pbs.twimg.com/media/HNFc4Dma8AA76FW.jpg?name=orig

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#88

Post by quotemstr »

noname120 wrote: post #73
quotemstr wrote: post #67

Codex is open source. You can build it yourself. You don't have to use upstream's parameter choices, compaction strategies, subagent heuristics, or whatever. Strange how people act like tweaks to these parameters are mandates when they're really just suggestions relayed to you via git. For example: upstream recently changed the effort level hotkeys so M-. would stop at xhigh, not max. If you want max, upstream, you g…

Not true. Try to increase the context window and you will be greeted with an error when you reach the real limit enforced server-side.

Perhaps not this for this parameter, but for others? Client side.

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#90

Post by QuantumGood »

tekacs wrote: post #26

I know a lot of people like to say that compaction makes this moot, but the level of detail you lose across compaction is wildly too much for most things that I do, unfortunately. Perhaps if your plans don't have as much detail, or if you're not, for example, having a discussion with a lot of nitty-gritty then it's fine? The lack of long context is the main reason that I still end up using Anthropic. The worst is whe…

Compaction is a feature like swapping out a car for a tricycle is a feature for greater economy.
Post reply on HN