Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

github.com

Post reply on HN
Print view View original

101–110 of 165 posts

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#101

Post by embedding-shape »

mdgld wrote: post #99
topynate wrote: post #89

Earlier quoted context omitted.

I've noticed them trying to creep into my writing. It doesn't help that I was a heavy em-dash user ten years before GPT-3.

I’ve always loved em dashes…very sad they’re a hallmark of LLM slop now. That and trios in arguments. Maybe I’m part LLM?

Some of us been writing texts on the public internet for decades, and humans invented machines trained on our texts, so suddenly the text we write now sounds like robots? Only way to win is to stop caring, let the people believe you're a LLM if so be it.

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#102

Post by mohsen1 »

nsingh2 wrote: post #16

Oh this seems bad, and is fairly easy to reproduce using codex cli. You give it a puzzle prompt that it has to reason about and solve, occasionally it will seemingly short circuit and think for exactly 516 tokens, and return the wrong result. When it ends up using 6000-8000 thinking tokens it returns the correct result. Maybe some issue with adaptive thinking? Another point for local models I guess, don't have to wor…

I have a philosophical problem with adaptive thinking. It’s a dumb guess for how much thinking budget to allocate ahead of thinking. At least in the context of LLMs there is probably no way of knowing how much thinking (token generation) is needed. The problem space is infinity vast, similarly of two prompts is not going to help any LLM decide how much thinning is needed. Models already stop thinking before hitting the thinking budget.

Why there is so much effort in making adaptive thinking happen and don’t we train models to produce the end of thinning token better?

Feels like a bandaid. We need models to be trained to do a reasonable amount of reasoning (no pub intended):

    reason

    estimate remaining uncertainty

    continue?

    reason more

    repeat

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#103

Post by andy99 »

mdgld wrote: post #99
topynate wrote: post #89

Earlier quoted context omitted.

I've noticed them trying to creep into my writing. It doesn't help that I was a heavy em-dash user ten years before GPT-3.

I’ve always loved em dashes…very sad they’re a hallmark of LLM slop now. That and trios in arguments. Maybe I’m part LLM?

This isn’t just em-dashes—it's the empty phrase that includes both whatever the contrastive construction is called and an “Honestly”. It might have been human written but the density of LLM flags is undeniable.

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#104

Post by andy99 »

embedding-shape wrote: post #101
mdgld wrote: post #99

Earlier quoted context omitted.

I’ve always loved em dashes…very sad they’re a hallmark of LLM slop now. That and trios in arguments. Maybe I’m part LLM?

Some of us been writing texts on the public internet for decades, and humans invented machines trained on our texts, so suddenly the text we write now sounds like robots? Only way to win is to stop caring, let the people believe you're a LLM if so be it.

LLM writing style is trained in by data labellers, it’s not just emergent behavior from being trained on internet texts.

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#105

Post by linzhangrun »

cbg0 wrote: post #82
linzhangrun wrote: post #81

Earlier quoted context omitted.

No, the decline in GPT-5.5's performance over the past few weeks is clearly noticeable.

Doesn't look like it: https://marginlab.ai/trackers/codex/

Thanks for sharing this project. Maybe I'm being subjective.

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#106

Post by embedding-shape »

andy99 wrote: post #104
embedding-shape wrote: post #101

Earlier quoted context omitted.

Some of us been writing texts on the public internet for decades, and humans invented machines trained on our texts, so suddenly the text we write now sounds like robots? Only way to win is to stop caring, let the people believe you're a LLM if so be it.

LLM writing style is trained in by data labellers, it’s not just emergent behavior from being trained on internet texts.

Ultimately it's a mix-match of everything, including whatever data the pre-training uses and how exactly they do the post-training. I don't think you can say there is a single factor that decides the writing style, unless you have some particular insight into some specific pipeline. Generally though, they output text that looks like the human text they ingested for training.

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#107

Post by lmwnshn »

miki123211 wrote: post #93
tunesmith wrote: post #34

Earlier quoted context omitted.

How do you mean OpenAI lets you use your own harness? I'm under the impression that a custom harness requires the OpenAI SDK, which requires api tokens rather than plus/pro accounts.

OpenAI doesn't require that; only Anthropic does. OpenAI's harness is fully open source[1], and (AFAIK) doesn't come with any kind of signed-build request integrity verification like Claude does. And by that logic, if you're allowed to use their API with a fork of Codex that you yourself compiled, there's nothing stopping you from making some other harness act like such a fork. [1] https://github.com/openai/codex

And although Codex Desktop is not open source (AFAIK), it does expose CDP and appserver which allowed me to build a Greasemonkey-like plugin system on top of it. It's surprisingly powerful, you can add transcript annotations, programmatically control the side panels, integrate a native-looking account switcher, and so on.

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#108

Post by AmazingTurtle »

It's funny, they sell you a subscription for frontier models, then over time begin to nerf them rapidly and no one talks about it. Should give me a discount when they reduce reasoning effort silently on the server side!

But on the other hand, I've been using 5.5-high on a daily basis in multithreading workflows, i.e. in parallel. I'm barely exhausting my weekly limits. I can't even Human-as-a-Service fast enough to catch up and read all the plans and implementations it does. So there is that.

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#109

Post by varispeed »

nsingh2 wrote: post #16

Oh this seems bad, and is fairly easy to reproduce using codex cli. You give it a puzzle prompt that it has to reason about and solve, occasionally it will seemingly short circuit and think for exactly 516 tokens, and return the wrong result. When it ends up using 6000-8000 thinking tokens it returns the correct result. Maybe some issue with adaptive thinking? Another point for local models I guess, don't have to wor…

And people pay for those wasted tokens? If that's the case, it is probably good idea to ask for refunds.

Re: GPT-5.5 Codex reasoning-token clustering may be leading to degraded performance

#110

Post by beacon294 »

miki123211 wrote: post #92
postalcoder wrote: post #59

Earlier quoted context omitted.

You still have to worry about misconfigured local models. Even the professionals get it wrong, which is why local model performance is uneven across providers.

And to add insult to injury, some providers will ride on the good reputation of some local model, selling you a terrible quant instead. With OpenAI, at least my gpt-5.5 is the same as your gpt-5.5. You can't say that about glm for example.

That's not a real equivalency. They are not necessarily the same (testing in production, hello!) And most importantly you do not have a local model because openai is not open!
Post reply on HN