Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unrolling the Codex agent loop

openai.com

Post reply on HN
Print view View original

161–170 of 220 posts

Re: Unrolling the Codex agent loop

#161

Post by arianvanp »

lifthrasiir wrote: post #119
kylequest wrote: post #103

Earlier quoted context omitted.

By the way, I reversed engineered the Claude Code binary and started sharing different code snippets (on twitter/bluesky/mastadon/threads). There's a lot of code there, so I'm looking for requests in terms of what part of the code to share and analyze what it's doing. One of the requests I got was about the LSP functionality in CC. Anything else you would find interesting to explore there? I'll post the whole thing i…

Not only this would violate the ToS, but also a newer native version of Claude Code precompiles most JS source files into the JavaScriptCore's internal bytecode format, so reverse engineering would soon become much more annoying if not harder.

Claude code is very good at reverse engineering. I reverse engineer Apple products in my MacBook all the time to debug issues

Re: Unrolling the Codex agent loop

#162

Post by xg15 »

e1g wrote: post #159
FuckButtons wrote: post #152

Earlier quoted context omitted.

They could be operating in latent space entirely maybe? It seems plausible to me that you can just operate on the embedding of the conversation and treat it as an optimization / compression problem.

Yes, Codex compaction is in the latent space (as confirmed in the article): > the Responses API has evolved to support a special /responses/compact endpoint [...] it returns an opaque encrypted_content item that preserves the model’s latent understanding of the original conversation

Is this what they mean by "encryption" - as in "no human-readable text"? Or are they actually encrypting the compaction outputs before sending them back to the client? If so, why?

Re: Unrolling the Codex agent loop

#163

Post by xg15 »

jumploops wrote: post #4

One thing that surprised me when diving into the Codex internals was that the reasoning tokens persist during the agent tool call loop, but are discarded after every user turn. This helps preserve context over many turns, but it can also mean some context is lost between two related user turns. A strategy that's helped me here, is having the model write progress updates (along with general plans/specs/debug/etc.) to…

I think it might be a good decision though, as it might keep the context aligned with what the user sees.

If the reasoning tokens where persisted, I imagine it would be possible to build up more and more context that's invisible to the user and in the worst case, the model's and the user's "understanding" of the chat might diverge.

E.g. image a chat where the user just wants to make some small changes. The model asks whether it should also add test cases. The user declines and tells the model to not ask about it again.

The user asks for some more changes - however, invisibly to the user, the model keeps "thinking" about test cases, but never telling outside of reasoning blocks.

So suddenly, from the model's perspective, a lot of the context is about test cases, while from the user's POV, it was only one irrelevant question at the beginning.

Re: Unrolling the Codex agent loop

#164

Post by e1g »

xg15 wrote: post #162
e1g wrote: post #159

Earlier quoted context omitted.

Yes, Codex compaction is in the latent space (as confirmed in the article): > the Responses API has evolved to support a special /responses/compact endpoint [...] it returns an opaque encrypted_content item that preserves the model’s latent understanding of the original conversation

Is this what they mean by "encryption" - as in "no human-readable text"? Or are they actually encrypting the compaction outputs before sending them back to the client? If so, why?

"encrypted_content" is just a poorly worded variable name that indicates the content of that "item" should be treated as an opaque foreign key. No actual encryption (in the cryptographic sense) is involved.

Re: Unrolling the Codex agent loop

#165

Post by xg15 »

e1g wrote: post #164
xg15 wrote: post #162

Earlier quoted context omitted.

Is this what they mean by "encryption" - as in "no human-readable text"? Or are they actually encrypting the compaction outputs before sending them back to the client? If so, why?

"encrypted_content" is just a poorly worded variable name that indicates the content of that "item" should be treated as an opaque foreign key. No actual encryption (in the cryptographic sense) is involved.

Ah, that makes more sense. Thanks!

Re: Unrolling the Codex agent loop

#166

Post by depr »

tern wrote: post #142
consumer451 wrote: post #137

Earlier quoted context omitted.

This is a fun thought experiment. I believe that we are now at the $5 Uber (2014) phase of LLMs. Where will it go from here? How much will a synthetic mid-level dev (Opus 4.5) cost in 2028, after the VC subsidies are gone? I would imagine as much as possible? Dynamic pricing? Will the SOTA model labs even sell API keys to anyone other than partners/whales? Why even that? They are the personalized app devs and hosts!…

While I remember $5 Ubers fondly, I think this situation is significantly more complex: - Models will get cheaper, maybe way cheaper - Model harnesses will get more complex, maybe way more complex - Local models may become competitive - Capital-backed access to more tokens may become absurdly advantaged, or not The only thing I think you can count on is that more money buys more tokens, so the more money you have, th…

If the electric grid cannot keep up with the additional demand, inference may not get cheaper. The cost of electricity would go up for LLM providers, and VCs would have to subsidize them more until the price of electricity goes down, which may take longer than they can wait, if they have been expecting LLM's to replace many more workers within the next few years.

Re: Unrolling the Codex agent loop

#167

Post by EnPissant »

FuckButtons wrote: post #151
consumer451 wrote: post #137

Earlier quoted context omitted.

This is a fun thought experiment. I believe that we are now at the $5 Uber (2014) phase of LLMs. Where will it go from here? How much will a synthetic mid-level dev (Opus 4.5) cost in 2028, after the VC subsidies are gone? I would imagine as much as possible? Dynamic pricing? Will the SOTA model labs even sell API keys to anyone other than partners/whales? Why even that? They are the personalized app devs and hosts!…

I can run Minimax-m2.1 on my m4 MacBook Pro at ~26 tokens/second. It’s not opus, but it can definitely do useful work when kept on a tight leash. If models improve at anything like the rate we have seen over the last 2 years I would imagine something as good as opus 4.5 will run on similarly specced new hardware by then.

I'm guessing that's ~26 decode tokens/s for 2-bit or 3-bit quantized Minimax-m2.1 at 0 context, and it only gets worse as the context grows.

I'm also sure your prefill is slow enough to make the model mostly unusable, even at smallish context windows, but entirely at mid to large context.

Re: Unrolling the Codex agent loop

#168

Post by swores »

mi_lk wrote: post #73
atonse wrote: post #68

Earlier quoted context omitted.

Probably because it doesn’t matter most of the time?

Same. If you're already using a proprietary model might as well just double down

But you don't have to be restricted to one model either? Codex being open source means you can choose to use Claude models, or Gemini, or...

It's fair enough to decide you want to just stick with a single provider for both the tool and the models, but surely still better to have an easy change possible even if not expecting to use it.

Re: Unrolling the Codex agent loop

#169

Post by EnPissant »

e1g wrote: post #164
xg15 wrote: post #162

Earlier quoted context omitted.

Is this what they mean by "encryption" - as in "no human-readable text"? Or are they actually encrypting the compaction outputs before sending them back to the client? If so, why?

"encrypted_content" is just a poorly worded variable name that indicates the content of that "item" should be treated as an opaque foreign key. No actual encryption (in the cryptographic sense) is involved.

Are you sure? For reasoning, encrypted_content is for sure actually encrypted.

Re: Unrolling the Codex agent loop

#170

Post by athrowaway3z »

frumplestlatz wrote: post #62
postalcoder wrote: post #56

The best part about this blog post is that none of it is a surprise – Codex CLI is open source. It's nice to be able to go through the internals without having to reverse engineer it. Their communication is exceptional, too. Eric Traut (of Pyright fame) is all over the issues and PRs. https://github.com/openai/codex

At this point I just assume Claude Code isn't OSS out of embarrassment for how poor the code actually is. I've got a $200/mo claude subscription I'm about to cancel out of frustration with just how consistently broken, slow, and annoying to use the claude CLI is.

I'm almost certain their code is a dumpster fire.

As for your 200$/mo sub. Dont buy it. If you read the fine print, their 20x usage is _per 5h session_, not overall usage.

Take 2x 100$ if you're hitting the limit.

Post reply on HN