Same price and a 4.5 bp jump from 72.7 to 77.2 SWEBench
Pretty solid progress for roughly 4 months.
Claude Sonnet 4.5
31–40 of 819 posts
Re: Claude Sonnet 4.5
#32Oh wow, a lot of focus on code from the big labs recently. In hindsight it makes sense that the domain the people building it know best is the one getting the most attention, and it's also the one the models have seen the most undeniable usefulness in so far. Though personally, the unpredictability of the future where all of this goes is a bit unsettling at the same time...
Re: Claude Sonnet 4.5
#33Price is playing a big role in my AI usage for coding. I am using Grok Code Fast as it's super cheap. Next to it GPT-5 Codex. If you are paying for model use out of pocket Claude prices are super expensive. With better tooling setup those less smart (and often faster) models can give you better results. I am going to give this another shot but it will cost me $50 just to try it on a real project :(
I'm paying $90(?) a month for the Max and it holds up for about an hour or so of in depth coding before it kicks in the 5-hour window lockout (so effectively about 4 hours of time when I can't run it). Kinda frustrating, even with efficient prompt and context length conservation techniques. I'm going to test this new sonnet 4.5, now but it'll probably be just as quick to gobble my credits.
Re: Claude Sonnet 4.5
#34That's a pretty pelican on a bicycle! https://jsbin.com/hiruvubona/edit?html,output https://claude.ai/share/618abbbf-6a41-45c0-bdc0-28794baa1b6c
I never understood the point of the pellican on a bicycle exercise:
LLMs coding agent doesnt have any way to see the output.
It means the only thing this test is testing, is the ability of the LLMs to memorise.
Edit: just to show my point, a regular human on a bicycle is way worse with the same model: https://i.imgur.com/flxSJI9.png
Re: Claude Sonnet 4.5
#35That's a pretty pelican on a bicycle! https://jsbin.com/hiruvubona/edit?html,output https://claude.ai/share/618abbbf-6a41-45c0-bdc0-28794baa1b6c
... but can it create an svg renderer for claude's site.
Re: Claude Sonnet 4.5
#36So Opus isn't recommended anymore? Bit confusing
Don't think I've ever preferred Opus to Sonnet
Re: Claude Sonnet 4.5
#37Looking at the chart here, it seems like Sonnet 4 was already better than GPT-5-codex in the SWE verified benchmark. However, my subjective personal experience was GPT-5-codex was far better at complex problems than Claude Code.
My subjective personal experience is the exact opposite of yours, GPT-5-codex is super slow and the results are mediocre at best. I would probably stop using AI for coding if I was forced to use GPT-5-codex.
Re: Claude Sonnet 4.5
#38This looks exciting. I hope they add this to Windsurf soon.
Re: Claude Sonnet 4.5
#39That's a pretty pelican on a bicycle! https://jsbin.com/hiruvubona/edit?html,output https://claude.ai/share/618abbbf-6a41-45c0-bdc0-28794baa1b6c
I never understood the point of the pellican on a bicycle exercise: LLMs coding agent doesnt have any way to see the output. It means the only thing this test is testing, is the ability of the LLMs to memorise. Edit: just to show my point, a regular human on a bicycle is way worse with the same model: https://i.imgur.com/flxSJI9.png
Memorise what exactly?
Re: Claude Sonnet 4.5
#40Looking at the chart here, it seems like Sonnet 4 was already better than GPT-5-codex in the SWE verified benchmark. However, my subjective personal experience was GPT-5-codex was far better at complex problems than Claude Code.
The Anthropic models have been vibe-coding tuned. They're beasts at simple python/ts programs, but they definitely fall apart with scientific/difficult code and large codebases. I don't expect that to change with the new Sonnet.
In my experience Gemini 2.5 Pro is the star when it comes to complex codebases. Give it a single xml from repomix and make sure to use the one at the aistudio.