Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

OpenAI reduces Codex Model Context Size from 372k to 272k

github.com

Post reply on HN
Print view View original

111–120 of 194 posts

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#111

Post by jubilanti »

p1esk wrote: post #85
jubilanti wrote: post #79

Earlier quoted context omitted.

The fact there is no way to disable auto-compaction and no way to go back in the conversation history to before a compact makes codex a no-go for me on any codebase > 5kloc. Even worse it seems to fire randomly at 10-20% of context left. So really you have 80% of 272k as real usable context. Compaction kills my sessions, it hallucinates and is worse than starting fresh. I've had enough times screaming at my computer…

[flagged]

I did find a strategy, thank you very much, and it was easy:

  npm uninstall -g @openai/codex && curl -fsSL https://claude.ai/install.sh | bash
This was the final straw that got me to downgrade my OpenAI subscription and move the bulk of my spending to Anthropic and Claude code.

OpenAI devs, if you're reading, this is how you lost a customer.

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#112

Post by no_no_no_yes »

jubilanti wrote: post #111
p1esk wrote: post #85

Earlier quoted context omitted.

[flagged]

I did find a strategy, thank you very much, and it was easy: npm uninstall -g @openai/codex && curl -fsSL https://claude.ai/install.sh | bash This was the final straw that got me to downgrade my OpenAI subscription and move the bulk of my spending to Anthropic and Claude code. OpenAI devs, if you're reading, this is how you lost a customer.

Claude does the same?

It asks you to create new chats (Claude Design) and Claude Code nudges you to start over too, I think `/compact` is optional though?

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#114

Post by eagerpace »

jmalicki wrote: post #61
tekacs wrote: post #47

Earlier quoted context omitted.

That isn't really a way around it for the kinds of things I'm talking about. The whole point is that getting all of the relevant context to even orchestrate it takes up most of the model's 272K.

For those kind of tasks, I do hierarchical planning, and then parts of the plan can focus one agent on keeping a smaller amount of pre-determined context when orchestrating. Think: A VP of engineering has some super high level goal: "build a new product to beat our competitor in this new market". It then filters down and various levels of reports build more and more detailed yet more narrow pieces of planning/orchest…

Yes, sometimes planning spans sessions too but I have even that broken down into stages where we start with the architecture and research, then save that, then define the milestones, then break down into the individual issues that the less sophisticated models can implement in a single session.

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#115

Post by salamo »

conradkay wrote: post #87
brandoncarl wrote: post #55

With purely quadratic attention, the cost of the token at 372K is 87% more than the token at 272K. There are attention mechanisms that help to mitigate this, but you can clearly see in the chart that Tibo posted that the attention mechanisms they are using are still scaling quadratically. This becomes an optimization problem for token t: min f[cost, quality loss] = compaction cost[t] + compaction quality loss[t] + to…

It's not quadratic attention, you get that curve from the input tokens going up linearly, since the graph is measuring cumulative cost at each token count. Basically for y=5 it's 5+4+3+2+1, or f(x) = x(x+1)/2 https://pbs.twimg.com/media/HNFc4Dma8AA76FW.jpg?name=orig

On the one hand, compacting at 300k saves OpenAI 40%. That's great.

On the other hand, $10 for 1M tokens still seems really high? It's not too hard to blow through that in an hour or two.

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#116

Post by hakanderyal »

yearolinuxdsktp wrote: post #66
hakanderyal wrote: post #54

I have rule files that guides the agent towards my coding standards, code style, house rules etc. They alone cost 60-80k tokens, and they are the backbone of my system that prevents slop. Pre 1M context, I had to build complicated tooling to re-include the relevant docs to the context upon compaction, which relied on unstable transcription file format, which was a pain to maintain. With 1M context I deleted all of th…

Try running your rule files through an LLM for optimization. 60k-80k tokens is massive. Funnily enough, most anti-slop skills I found are both way too verbose and miss some common slop constructs. I also reduced many rules from “When doing X, don’t do Y, but do Z.” Instead, the rule is “When doing X, do Z.” Fewer tokens and often works better. I had one critical rule I was maintaining about searching the codebase usi…

It’s optimized to death. Every word, every comma. I have a rule file for writing rule files.

Problem stems from using my own abstractions instead of common frameworks.

For a full stack session, backend + front end + docs about the part we are working on adds to that amount.

I’ve accepted it because it results in code exactly the style I would’ve written. It’s a good tradeoff to cut off the slop.

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#118

Post by CamperBob2 »

simonw wrote: post #24
ashu1461 wrote: post #14

I am wondering why the codex repo is open source ? Can they afford it to be open source in the longer run ?

Not much point making it closed source at this point, coding agent harnesses are trivial to reverse-engineer. Being open source was also a useful community lever they could pull when they were trying to catch up with Claude Code.

Not much point making anything closed source at this point, if the client has access to the binary.

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#119

Post by Supermancho »

thejazzman wrote: post #62
jmalicki wrote: post #58

Earlier quoted context omitted.

Instead of compaction, I very aggressively do everything with teams of subagents. The main agent keeps the high level context, and all of the nitty gritty detail and code details not needed by the main agent never even make it into the context.

This used to be really effective and cheap, now it lights my quota on fire, even after I fix the config and restore smaller subagents instead of like-agents Multiple (slow) conversations seems to be the efficient path for me the past week or so

That is how the majority of developers use AI. Aim small miss small and all that. What's great about AI is that you can ask many many small questions about big data dumps (like logs and stack traces), for relatively little usage.

Re: OpenAI reduces Codex Model Context Size from 372k to 272k

#120

Post by jwitthuhn »

no_no_no_yes wrote: post #112
jubilanti wrote: post #111

Earlier quoted context omitted.

I did find a strategy, thank you very much, and it was easy: npm uninstall -g @openai/codex && curl -fsSL https://claude.ai/install.sh | bash This was the final straw that got me to downgrade my OpenAI subscription and move the bulk of my spending to Anthropic and Claude code. OpenAI devs, if you're reading, this is how you lost a customer.

Claude does the same? It asks you to create new chats (Claude Design) and Claude Code nudges you to start over too, I think `/compact` is optional though?

Claude does the same when you hit the context limit, but the difference is that claude's context limit is 1m and Codex's is 272k now. Claude lets you get a lot more done before you have to worry about the start getting compacted away.
Post reply on HN