[flagged]
OpenAI reduces Codex Model Context Size from 372k to 272k
51–60 of 194 posts
Re: OpenAI reduces Codex Model Context Size from 372k to 272k
#52[dead]
Re: OpenAI reduces Codex Model Context Size from 372k to 272k
#53I know a lot of people like to say that compaction makes this moot, but the level of detail you lose across compaction is wildly too much for most things that I do, unfortunately. Perhaps if your plans don't have as much detail, or if you're not, for example, having a discussion with a lot of nitty-gritty then it's fine? The lack of long context is the main reason that I still end up using Anthropic. The worst is whe…
That seems quite different from my design process. I write a plan.md that goes through multiple revisions. The plan is the memory. Restarting and reading the plan again to do another review is a good way to get a different perspective.
So far I have three different folders in my project for three different types of md documents that I keep creating and revising together with the agent: reports, plans, and reviews
Every large feature has at least one of each file created for it in that order. I start by analysing stuff and making reports that might also contain design thinking and decisions. Once I'm satisfied we have properly analysed and thought though whatever it is we're working on then I might make a plan or just jump into implementation depending on how complex the issue is. And then I might have zero or multiple reviews of different kinds by different agents depending on how much verification I think the issue warrants.
This is my super simple worse-is-better development process. I make up what is needed on the spot.
Re: OpenAI reduces Codex Model Context Size from 372k to 272k
#54I have rule files that guides the agent towards my coding standards, code style, house rules etc. They alone cost 60-80k tokens, and they are the backbone of my system that prevents slop. Pre 1M context, I had to build complicated tooling to re-include the relevant docs to the context upon compaction, which relied on unstable transcription file format, which was a pain to maintain. With 1M context I deleted all of those. Nowadays most of my sessions uses 300-450k context.
Another thing that's preventing me from trying Codex. (the other is @ referencing files not auto including them to the context)
1M should be table stakes for frontier models at this point for programming.
Re: OpenAI reduces Codex Model Context Size from 372k to 272k
#55With purely quadratic attention, the cost of the token at 372K is 87% more than the token at 272K.
There are attention mechanisms that help to mitigate this, but you can clearly see in the chart that Tibo posted that the attention mechanisms they are using are still scaling quadratically.
This becomes an optimization problem for token t:
min f[cost, quality loss] = compaction cost[t] + compaction quality loss[t] + token cost[t]
Re: OpenAI reduces Codex Model Context Size from 372k to 272k
#56I know a lot of people like to say that compaction makes this moot, but the level of detail you lose across compaction is wildly too much for most things that I do, unfortunately. Perhaps if your plans don't have as much detail, or if you're not, for example, having a discussion with a lot of nitty-gritty then it's fine? The lack of long context is the main reason that I still end up using Anthropic. The worst is whe…
> The lack of long context is the main reason that I still end up using Anthropic.
Anthropic's "long context" is mostly marketing bullshit. It semi-reliably holds about the same amount of info, and then starts suffering the same issues.
Re: OpenAI reduces Codex Model Context Size from 372k to 272k
#57I know a lot of people like to say that compaction makes this moot, but the level of detail you lose across compaction is wildly too much for most things that I do, unfortunately. Perhaps if your plans don't have as much detail, or if you're not, for example, having a discussion with a lot of nitty-gritty then it's fine? The lack of long context is the main reason that I still end up using Anthropic. The worst is whe…
Why arent you using /goal?
Re: OpenAI reduces Codex Model Context Size from 372k to 272k
#58I know a lot of people like to say that compaction makes this moot, but the level of detail you lose across compaction is wildly too much for most things that I do, unfortunately. Perhaps if your plans don't have as much detail, or if you're not, for example, having a discussion with a lot of nitty-gritty then it's fine? The lack of long context is the main reason that I still end up using Anthropic. The worst is whe…
Instead of compaction, I very aggressively do everything with teams of subagents. The main agent keeps the high level context, and all of the nitty gritty detail and code details not needed by the main agent never even make it into the context.
Re: OpenAI reduces Codex Model Context Size from 372k to 272k
#59I never felt that context size was any issue at all in codex. I don't know how their compaction works, but it just keeps on going as if it has no context limit at all. At least in my experience.
Most problems should be divide-and-conquerable into chunks that 300 vs 400k context is basically never an issue. A coding agent isn't an infinite chat.
Re: OpenAI reduces Codex Model Context Size from 372k to 272k
#60"272k ought to be enough for anybody"
Thats not even enough to read a simple codebase, how is that enough?
Almost certainly an homage to “640kB ought to be enough for anybody.”