861–870 of 1001 posts
It also defaults to 'low' mode for some reason. Can't tell if that's a step backwards compared to GPT-5.5 in medium mode so I'm sticking to medium.
Edit: just noticed it's spawning subagents in 'high' thinking mode.
GPT-5.6 Sol sets a new SOTA on ARC-AGI-3: 7.8% Sol is the first verified frontier model to ever beat an ARC-AGI-3 game https://arcprize.org/results/openai-gpt-5-6
We have it slightly ahead of Fable in our multi-agent coding evaluations. Fable's main advantage is that its average solution size is smaller. However, GPT 5.6 Sol is a substantial improvement from GPT 5.4/5.5 which would write verbose, defensive code. 31KB for GPT 5.4/5.5 down to 26KB for GPT 5.6 Sol, with better performance for Sol. Fable scores slightly lower, but with an average solution size of 12.2 KB. Data at…
I wonder why nobody has tried to optimize for actual code size or complexity metric, or at least why I haven’t seen more benchmarks that display this. GPT5.5 just keeps pushing more and more pointless indirection into every function it writes in my main project, it’s borderline negative productivity.
P.S. I’d be curious to see Cursor’s composer models in there, they seem to be among the best performing low cost models: https://artificialanalysis.ai/articles/cursor-composer-2-5-c...
Earlier quoted context omitted.
Now we have various Opus+ level models (Opus/Fable, Grok 4.5, GPT 5.6) I prefer to focus on price/speed and harness as models are all generally good enough for coding. (Fable is overkill for 90% of work but is still level above). So I use Grok Build with 4.5 as its VERY fast and cheap, Codex is next best for me with sol/lunar 5.6. and Claude Code Fable for the 10% of tasks that need that level of reasoning. However I…
Am I missing something or isn't sol/lunar 5.6 only out for like 3 hours? How did you evaluate?
They talk a lot about speed in the article, but having tried out Sol today with Pi, 'medium' mode, one thing that stands out is that it's really ssslllloooowww. It also defaults to 'low' mode for some reason. Can't tell if that's a step backwards compared to GPT-5.5 in medium mode so I'm sticking to medium. Edit: just noticed it's spawning subagents in 'high' thinking mode.
Earlier quoted context omitted.
Looks like they reset everyone's Fable usage.
They did. I wonder if Anthropic will also be removing the 50% limit. My Fable weekly limit is at 15% used already, 5.6 Sol at 3% used. And this is with the Max 20x plan compared to Codex 5x. I don't work on the same tasks to compare them objectively, but GPT 5.6 on xhigh seems much cheaper. Essentially unlimited usage.
The gap between Opus 4.8 and Fable is large enough to drive a GPT 5.6 sized bus through. A better Opus would take some of the heat off.
Weirdly, normally new ChatGPT releases are head and shoulders above anything else, but according to OpenAI's own evaluation, Anthropic's Mythos outperforms ChatGPT in quite a few benchmarks: https://openai.com/index/gpt-5-6/ . ChatGPT 6 must be deep in the pipeline and will be released within the next few months. Maybe that's why this release is versioned 5.6, not 6.0.
Earlier quoted context omitted.
> can better infer the user’s underlying goal and intended level of work This is a trap. It's the optimistic fallacy that poisons all "consumer scale" machine learning products and what's going to effectively ruin these models as they keep chasing it in the same way that web queries were ruined, social media feeds were ruined, and media recommenders were ruined. For the vendor, optimizing metrics across their whole u…
I want my model to help me build up its own infrastructure that instills it with the sort of constraints I want for my project, rather than have it behave generically and automatically for everything. It should follow instructions incredibly well while inferring contradictions or gaps in logic and surfacing those to the user as suggestions for improvements and persistence. I really hate how Claude just assumes you wa…
I want the same as you, and even further, I want a model that refuses to execute changes I request if they don't make sense considering the context, or if they're impossible, and avoid any sort of quick hacks and patches. But I also want a model that does the pure opposite, that I can chuck a "Do X" query at and it figures it out. Then I'm sure there are middle-zones between these two, or even more extremes too.
But the choice isn't there, we get to chose between "fast/stupid", "medium/medium" and "slow/smart", then that's it. With system prompts we get to steer it a bit, but I've needed to make my own fork of codex to surface those things to me (the user) so I can control it better, and different models respond differently to the "Stop and don't implement anything if the request doesn't make sense yadda yadda" parts, would be lovely to have those sort of "personalities" surfaced up front when making decisions about what model to use.
Ok long time Claude Code user here; lately I've started to realize there's other great models out there I should be trying, but I'm hesitant to leave Claude Code behind for something new. What's the consensus today on codex vs claude code, does it really matter anymore?