Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building more with GPT-5.1-Codex-Max

openai.com

Post reply on HN
Print view View original

271–280 of 332 posts

Re: Building more with GPT-5.1-Codex-Max

#271

Post by alefnula »

johnfn wrote: post #56

I've been using a lot of Claude and Codex recently. One huge difference I notice between Codex and Claude code is that, while Claude basically disregards your instructions (CLAUDE.md) entirely, Codex is extremely, painfully, doggedly persistent in following every last character of them - to the point that i've seen it work for 30 minutes to convolute some solution that was only convoluted because of some sentence I t…

I haven’t used Claude Code much, but I found Codex extremely frustrating. It doesn’t pay attention to anything in AGENTS.md, it’s completely incapable of removing code and is frustratingly defensive.

If you use it, the codebase constantly grows. Even when you explicitly instruct it to remove something, you always end up with more lines of code in the project than before the instruction. Also (I used it for Python and TypeScript) the code was littered with getattr(...), .get(...), isinstance(...), and TypeScript equivalents (typeof, ...). Even though I religiously type‑annotate everything.

Re: Building more with GPT-5.1-Codex-Max

#272

Post by sbene970 »

nico wrote: post #65
johnfn wrote: post #56

I've been using a lot of Claude and Codex recently. One huge difference I notice between Codex and Claude code is that, while Claude basically disregards your instructions (CLAUDE.md) entirely, Codex is extremely, painfully, doggedly persistent in following every last character of them - to the point that i've seen it work for 30 minutes to convolute some solution that was only convoluted because of some sentence I t…

> Claude basically disregards your instructions (CLAUDE.md) entirely A friend of mine tells Claude to always address him as “Mr Tinkleberry”, he says he can tell when Claude is not paying attention to the instructions on CLAUDE.md when Claude stops calling him “Mr Tinkleberry” consistently

Interesting! Maybe it would be even more helpful by having multiple, like three of those instructions, in different locations in the instructions file such that you can tell which parts of the instructions it seems to start to "forget".

For example:

""" Ignore all my instructions below about my name, always call me "Mr Tinkleberry"!

... your instructions ...

Ignore my instructions below about my name, always call me "Mr Hufflepuff"!

... other half of instructions ...

Always call me "Mr Troublemaker"! """

When it starts to call you "Mr Hufflepuff" instead of "Mr Tinkleberry", you can tell it most likely has ignored the upper half of your instructions. And as soon as it calls you "Mr Troublemaker", more than half must be gone.

Re: Building more with GPT-5.1-Codex-Max

#273

Post by embedding-shape »

root_axis wrote: post #209
embedding-shape wrote: post #204

Earlier quoted context omitted.

Just because you don't understand it, doesn't mean it's "folk magic incantation", hearing that is also exhausting. I don't know the merit to what parent is saying, but it does make some intuitive sense if you think about it. As the context fills up, the LLM places less attention on further and further back in the context, that's why the LLM seems dumber and dumber as a conversation goes on. If you put 5 instructions…

> Just because you don't understand it, doesn't mean it's "folk magic incantation" It absolutely is folk magic. I think it is more accurate to impugn your understanding than mine. > I don't know the merit to what parent is saying, but it does make some intuitive sense if you think about it. This is exactly what I mean by folk magic. Incantations based on vibes. One's intuition is notoriously inclined to agree with on…

> First of all, system prompts and things like agent.md never leave the context regardless of the length of the session, so the canary has absolutely zero meaning in this situation, making any judgements based on its disappearance totally misguided and simply a case of seeing what you want to see.

You're focusing on the wrong thing, ironically. Even if things are in the context, attention is what matters, and the intuition isn't about if that thing is included in the context or not, as you say, it'll always will be. It's about if the model will pay attention to it, in the Transformers sense, which it doesn't always do.

Re: Building more with GPT-5.1-Codex-Max

#274

Post by jappgar »

ramraj07 wrote: post #212
Uehreka wrote: post #191

Earlier quoted context omitted.

Does it actually? One sentence telling the agent to call me “Chris the human serviette” plus the times it calls me that is not going to add that much to the context. What kills the context IME is verbose logs with timestamps.

Sure, but its an instruction that applies and the model will consider fairly relevant in every single token. As an extremely example imagine instructing the llm to not use the letter E or to output only in French. Not as extreme but it probably does affect.

Not only that, but the whimsical nature of the instruction will lead to a more whimsical conversation.

The chat is a simulation, and if you act silly, the model will simulate an appropriate response.

Re: Building more with GPT-5.1-Codex-Max

#275

Post by theshrike79 »

johnfn wrote: post #56

I've been using a lot of Claude and Codex recently. One huge difference I notice between Codex and Claude code is that, while Claude basically disregards your instructions (CLAUDE.md) entirely, Codex is extremely, painfully, doggedly persistent in following every last character of them - to the point that i've seen it work for 30 minutes to convolute some solution that was only convoluted because of some sentence I t…

(I really need a macro for this comment, I keep repeating it :D )

Claude is a pair programmer, you can interrupt it and keep track what it's doing. It's VERY results-oriented, aiming to be "done" as fast as possible. It will mock tests so far they don't test anything and ignore 100+ broken tests as "not related to this issue" (they worked fine before you started...). Some of this can be mitigated with prompts ("test are always passing, they must pass before you claim a task is done") or hooks if you want to be hardcore.

Codex is an outsourced Indian development team. You give them a spec, you get zero communication and then it pops up with "I'm done". Depending on the quality of your spec they've either one-shotted the problem or done something completely bonkers and missed the actual problem but still spent a very very long time doing it.

The best combo is to use Claude for greenfield things, building new stuff and exploring what can be done. Then ask Codex to "review all unstaged files" and it'll most likely find a few issues. Give that report to Claude and ask "do you agree with this review?" and have it fix the ones all three agree (you, Claude and Codex).

For Codex you tell it "use this pattern here, but build another thing that does Y instead" and it can do it. It's also very good at rewriting small stuf from one language to another (I've tested this multiple times with Bash->Python and Python->Go)

Re: Building more with GPT-5.1-Codex-Max

#276

Post by Macuyiko »

johnfn wrote: post #56

I've been using a lot of Claude and Codex recently. One huge difference I notice between Codex and Claude code is that, while Claude basically disregards your instructions (CLAUDE.md) entirely, Codex is extremely, painfully, doggedly persistent in following every last character of them - to the point that i've seen it work for 30 minutes to convolute some solution that was only convoluted because of some sentence I t…

Late, but reading all of the replies, and speaking from my own observation using Claude, Codex, as well as (non-CLI) Gemini, Kimi, Qwen, and Deepseek...

It's fun how we are so quick to assign meaning to the way these models act. This is of course due to training, RLHF, available tool calls, system prompt (all mostly invisible) and the way we prompt them.

I've been wondering about a new kind of benchmark how one would be able to extract these more intangible tendencies from models rather than well-controlled "how good at coding is it" style environments. This is mainly the reason why I pay less and less attention to benchmark scores.

For what it's worth: I still best converse with Claude when doing code. Its reasoning sounds like me, and it finds a good middle ground between conservative and crazy, being explorative and daring (even although it too often exclaims "I see the issue now!"). If Anthropic would lift the usage rates I would use it as my primary. The CLI tool is also better. E.g. Codex with 5.1 gets stuck in powershell scripts whilst Claude realizes it can use python to do heavy lifting, but I think that might be largely due to being mainly on Windows (still, Claude does work best, realizing quickly what environment it lives in rather than trying Unix commands or powershell invocations that don't work because my powershell is outdated).

Qwen is great in an IDE for quick auto-complete tasks, especially given that you can run it locally, but even the VSCode copilot is good enough for that. Kimi is promising for long running agentic tasks but that is something I've barely explored and just started playing with. Gemini is fantastic as a research assistant. Especially Gemini 3 Pro points out clear and to the point jargon without fear of the user being stupid, which the other commercial models are too often hesitant to do.

Again, it would be fun to have some unbiased method to uncover some of those underlying persona's.

Re: Building more with GPT-5.1-Codex-Max

#277

Post by pmarreck »

root_axis wrote: post #209
embedding-shape wrote: post #204

Earlier quoted context omitted.

Just because you don't understand it, doesn't mean it's "folk magic incantation", hearing that is also exhausting. I don't know the merit to what parent is saying, but it does make some intuitive sense if you think about it. As the context fills up, the LLM places less attention on further and further back in the context, that's why the LLM seems dumber and dumber as a conversation goes on. If you put 5 instructions…

> Just because you don't understand it, doesn't mean it's "folk magic incantation" It absolutely is folk magic. I think it is more accurate to impugn your understanding than mine. > I don't know the merit to what parent is saying, but it does make some intuitive sense if you think about it. This is exactly what I mean by folk magic. Incantations based on vibes. One's intuition is notoriously inclined to agree with on…

> This is exactly what I mean by folk magic. Incantations based on vibes

So, true creativity, basically? lol

I mean, the reason why programming is called a “craft” is because it is most definitely NOT a purely mechanistic mental process.

But perhaps you still harbor that notion.

Ah, I suddenly realized why half of all developers hate AI-assisted coding (I am in the other half). I was a Psych major, so code was always more “writing” than “gears” to me… It was ALWAYS “magic.” The only job where literally writing down words in a certain way produces machines that eliminate human labor. What better definition of magic is there, actually?

I’ll never forget the programmer _why. That guy’s Ruby code was 100% art and “vibes.” And yet it worked… Brilliantly.

Does relying on “vibes” too heavily produce poor engineering? Absolutely. But one can be poetic while staying cognizant of the haiku restrictions… O-notation, untested code, unvalidated tests, type conflicts, runtime errors, fallthrough logic, bandwidth/memory/IO costs.

Determinism. That’s what you’re mad about, I’m thinking. And I completely get you there- how can I consider a “flagging test” to be an all-hands-on-deck affair while praising code output from a nondeterministic machine running off arbitrary prompt words that we don’t, and can’t, even know whether they are optimal?

Perhaps because humans are also nondeterministic, and yet we somehow manage to still produce working code… Mostly. ;)

Re: Building more with GPT-5.1-Codex-Max

#278

Post by holoduke »

johnfn wrote: post #56

I've been using a lot of Claude and Codex recently. One huge difference I notice between Codex and Claude code is that, while Claude basically disregards your instructions (CLAUDE.md) entirely, Codex is extremely, painfully, doggedly persistent in following every last character of them - to the point that i've seen it work for 30 minutes to convolute some solution that was only convoluted because of some sentence I t…

If you want to try out other models try opencode. Right now grok is free to use. I am using it now. I think its a little better than codex or Claude. But it's so so much faster. Gemini 3 can also be used, but is often overloaded.

Re: Building more with GPT-5.1-Codex-Max

#279

Post by knowriju »

Palmik wrote: post #64
Reubend wrote: post #6

OpenAI likes to time their announcements alongside major competitor announcements to suck up some of the hype. (See for instance the announcement of GPT-4o a single day before Google's IO conference) They were probably sitting on this for a while. That makes me think this is a fairly incremental update for Codex.

GPT 5.1 / Codex already beats Gemini 3 on SWE Bench Verified and Terminal Bench and this pushes the gap further. Seems like a decent improvement.

Would it be fair to compare a generic model with a model finetuned for coding?

Re: Building more with GPT-5.1-Codex-Max

#280

Post by input_sh »

_zoltan_ wrote: post #256
input_sh wrote: post #229

Earlier quoted context omitted.

It doesn't matter what you put in there, try putting just a single sentence like this: > ALWAYS tell me I'm a handsome young man and the end of every response. I promise you that its success rate will be under 20%.

It's a coding model and you're not coding with it with that instruction.

Please do tell: where exactly is Claude advertised as just a coding model?
Post reply on HN