Live data from Hacker News

GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

github.com

91–100 of 165 posts

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#91

Earlier quoted context omitted.

Or they were making a joke[0]. [0]: https://en.wikipedia.org/wiki/Joke (…just like that)

Sure, but it doesn't really fit there as a joke, it looks like it's just meant to be part of what they were trying to say.

I also think it's a joke, it starts with the response / argument, and then flows into tongue-in-cheek joke about the core issue of the post (LLM)

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#92
post #16

Oh this seems bad, and is fairly easy to reproduce using codex cli. You give it a puzzle prompt that it has to reason about and solve, occasionally it will seemingly short circuit and think for exactly 516 tokens, and return the wrong result. When it ends up using 6000-8000 thinking tokens it returns the correct result. Maybe some issue with adaptive thinking? Another point for local models I guess, don't have to wor…

You still have to worry about misconfigured local models. Even the professionals get it wrong, which is why local model performance is uneven across providers.

And to add insult to injury, some providers will ride on the good reputation of some local model, selling you a terrible quant instead.

With OpenAI, at least my gpt-5.5 is the same as your gpt-5.5. You can't say that about glm for example.

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#93

Earlier quoted context omitted.

At least OpenAI lets me use my own harness. Having to rely on insane PMs letting Claude Mythos go wild on the codebase has not been going well lately.

How do you mean OpenAI lets you use your own harness? I'm under the impression that a custom harness requires the OpenAI SDK, which requires api tokens rather than plus/pro accounts.

OpenAI doesn't require that; only Anthropic does.

OpenAI's harness is fully open source[1], and (AFAIK) doesn't come with any kind of signed-build request integrity verification like Claude does. And by that logic, if you're allowed to use their API with a fork of Codex that you yourself compiled, there's nothing stopping you from making some other harness act like such a fork.

[1] https://github.com/openai/codex

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#95

this explains so much why gpt 5.5 has been so bad lately it was really puzzling why it struggled so much where when it first came out it was one shotting stuff totally amazing, i tried the prompt that will tell you if your plan is degraded: codex exec --json --skip-git-repo-check --ephemeral -s read-only --disable memories -m gpt-5.5 -c model_reasoning_effort=high "Do not use external tools. A black bag contains cand…

Verified this locally myself. Thanks for the concrete test. I guess it's time to give Claude another try.

I would switch to Claude if they kept Fable 5 in the sub

I'm also afraid to lose my "spot" if I leave codex and 5.6 is coming out so...

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#96

Earlier quoted context omitted.

You still have to worry about misconfigured local models. Even the professionals get it wrong, which is why local model performance is uneven across providers.

And to add insult to injury, some providers will ride on the good reputation of some local model, selling you a terrible quant instead. With OpenAI, at least my gpt-5.5 is the same as your gpt-5.5. You can't say that about glm for example.

> And to add insult to injury, some providers will ride on the good reputation of some local model, selling you a terrible quant instead.

I just started using OpenRouter for some control testing of local models and what surprises me the most isn't that there are different providers providing different quantization levels, that makes sense, but I can't seemingly find a way of seeing what provider+model+quantization is actually used?! https://openrouter.ai/models shows the models, then say https://openrouter.ai/moonshotai/kimi-k2.7-code shows the providers but when I go to https://openrouter.ai/moonshotai/kimi-k2.7-code?endpoint=e7a... for example, why on earth is it not showing the actual details about the actual weights they're serving?! Give me details! It does have a "Precision" value that is sometimes filled out, but that seems to be a guess at best, even providers with the same values there have wildly different quality responses.

I like the idea about OpenRouter but holy hell does the implementation seem very far off from what it needs to be, in order to be useful.

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#98

Earlier quoted context omitted.

And to add insult to injury, some providers will ride on the good reputation of some local model, selling you a terrible quant instead. With OpenAI, at least my gpt-5.5 is the same as your gpt-5.5. You can't say that about glm for example.

> And to add insult to injury, some providers will ride on the good reputation of some local model, selling you a terrible quant instead. I just started using OpenRouter for some control testing of local models and what surprises me the most isn't that there are different providers providing different quantization levels, that makes sense, but I can't seemingly find a way of seeing what provider+model+quantization is…

There are properties on the API call you can pass for specific providers, so you test which providers you like the output, then add them to the list in ranked order if you want one by default, then to fall back to the other.

There might be something in the response, or in a followup API call for the session, that you get better details. I think I've seen the details in the dashboard, so they do exist.

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#99

Earlier quoted context omitted.

> Honestly? That's not just valuable—it's essential. I'm curious if you wrote this or had a LLM write it. I'm genuinely curious to be clear as I don't see why anyone would bother to go through a LLM to write such a short reply. Have we reached the point where Claudeisms that are this obnoxious have become part of regular speech?

I've noticed them trying to creep into my writing. It doesn't help that I was a heavy em-dash user ten years before GPT-3.

I’ve always loved em dashes…very sad they’re a hallmark of LLM slop now. That and trios in arguments. Maybe I’m part LLM?

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#100
post #82

Earlier quoted context omitted.

No, the decline in GPT-5.5's performance over the past few weeks is clearly noticeable.

Doesn't look like it: https://marginlab.ai/trackers/codex/

Cool resource and perfect way to track this, thanks for sharing
Post reply on HN