Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Sonnet 4.5

anthropic.com

Post reply on HN
Print view View original

41–50 of 819 posts

Re: Claude Sonnet 4.5

#41

Post by AnotherGoodName »

yewenjie wrote: post #5

Looking at the chart here, it seems like Sonnet 4 was already better than GPT-5-codex in the SWE verified benchmark. However, my subjective personal experience was GPT-5-codex was far better at complex problems than Claude Code.

I always wonder how absolute in performance a given model is. Sometimes i ask for Claude-Opus and the responses i get back are worse than the lowest end models of other assistants. Other times it surprises me and is clearly best in class.

Sometimes in between this variability of performance it pops up a little survey. "How's Claude doing this session from 1-5? 5 being great." and i suspect i'm in some experiment of extremely low performance. I'm actually at the point where i get the feeling peak hour weekdays is terrible and odd hour weekends are great even when forcing a specific model.

While there is some non-determinism it really does feel like performance is actually quite variable. It would make sense they scale up and down depending on utilization right? There was a post a week ago from Anthropic acknowledging terrible model performance in parts of August due to an experiemnt. Perhaps also at peak hour GPT has more datacenter capacity and doesn't get degraded as badly? No idea for sure but it is frustrating when simple asks fail and complex asks succeed without it being clear to me why that may be.

Re: Claude Sonnet 4.5

#42

Post by _joel »

Kuinox wrote: post #34
mohsen1 wrote: post #19

That's a pretty pelican on a bicycle! https://jsbin.com/hiruvubona/edit?html,output https://claude.ai/share/618abbbf-6a41-45c0-bdc0-28794baa1b6c

I never understood the point of the pellican on a bicycle exercise: LLMs coding agent doesnt have any way to see the output. It means the only thing this test is testing, is the ability of the LLMs to memorise. Edit: just to show my point, a regular human on a bicycle is way worse with the same model: https://i.imgur.com/flxSJI9.png

Because it excercises thinking about a pelican riding a bike (not common) and then describing that using SVG. It's quite nice imho and seems to scale with the power of the LLM model. Sure Simon has some actual reasons though.

Re: Claude Sonnet 4.5

#43

Post by TrainedMonkey »

cellis wrote: post #21
yewenjie wrote: post #5

Looking at the chart here, it seems like Sonnet 4 was already better than GPT-5-codex in the SWE verified benchmark. However, my subjective personal experience was GPT-5-codex was far better at complex problems than Claude Code.

Opposite for me…5-codex high ran out of tokens extremely quickly and didn’t adhere as well to the agents.md as Claude did to the Claude.md, perhaps because it insists on writing extremely complicated bash scripts or whole python programs to execute what should be simple commands.

Codex was a miserable experience for me until I learned to compact after every feature. Now it is a cut above CC, although the latter still has an edge at TODO scaffolding and planning.

Re: Claude Sonnet 4.5

#44

Post by esafak »

muttantt wrote: post #18
mohsen1 wrote: post #14

Price is playing a big role in my AI usage for coding. I am using Grok Code Fast as it's super cheap. Next to it GPT-5 Codex. If you are paying for model use out of pocket Claude prices are super expensive. With better tooling setup those less smart (and often faster) models can give you better results. I am going to give this another shot but it will cost me $50 just to try it on a real project :(

how are you using grok code fast? what tooling/cli/etc?

Through Opencode.

Re: Claude Sonnet 4.5

#45

Post by hu3 »

muttantt wrote: post #18
mohsen1 wrote: post #14

Price is playing a big role in my AI usage for coding. I am using Grok Code Fast as it's super cheap. Next to it GPT-5 Codex. If you are paying for model use out of pocket Claude prices are super expensive. With better tooling setup those less smart (and often faster) models can give you better results. I am going to give this another shot but it will cost me $50 just to try it on a real project :(

how are you using grok code fast? what tooling/cli/etc?

free in GitHub copilot atm

Re: Claude Sonnet 4.5

#46

Post by cube2222 »

So… seems like we’re back to Sonnet being better than Opus? At least based on their benchmarks.

Curious to see that in practice, but great if true!

Re: Claude Sonnet 4.5

#47

Post by lordnacho »

jasonsb wrote: post #37
yewenjie wrote: post #5

Looking at the chart here, it seems like Sonnet 4 was already better than GPT-5-codex in the SWE verified benchmark. However, my subjective personal experience was GPT-5-codex was far better at complex problems than Claude Code.

My subjective personal experience is the exact opposite of yours, GPT-5-codex is super slow and the results are mediocre at best. I would probably stop using AI for coding if I was forced to use GPT-5-codex.

I'm on your side.

I find there's a quite large spread in ability between various models. Claude models seem to work superbly for me, though I'm not sure whether that's just a quirk of what my projects look like.

Re: Claude Sonnet 4.5

#48

Post by richwater »

AnotherGoodName wrote: post #41
yewenjie wrote: post #5

Looking at the chart here, it seems like Sonnet 4 was already better than GPT-5-codex in the SWE verified benchmark. However, my subjective personal experience was GPT-5-codex was far better at complex problems than Claude Code.

I always wonder how absolute in performance a given model is. Sometimes i ask for Claude-Opus and the responses i get back are worse than the lowest end models of other assistants. Other times it surprises me and is clearly best in class. Sometimes in between this variability of performance it pops up a little survey. "How's Claude doing this session from 1-5? 5 being great." and i suspect i'm in some experiment of e…

They absolutely mess with it

Re: Claude Sonnet 4.5

#49

Post by catigula »

I happened to be in the middle of a task in a production codebase that the various models struggled on so I can give a quick vibe benchmark:

opus 4.1: made weird choices, eventually got to a meh solution i just rolled back.

codex: took a disgusting amount of time but the result was vastly superior to opus. night and day superiority. output was still not what i wanted.

sonnet 4.5: not clearly better than opus. categorically worse decision-making than codex. very fast.

Codex was night and day the best. Codex scares me, Claude feels like a useful tool.

Re: Claude Sonnet 4.5

#50

Post by AlecSchueler »

greenfish6 wrote: post #28
mohsen1 wrote: post #19

That's a pretty pelican on a bicycle! https://jsbin.com/hiruvubona/edit?html,output https://claude.ai/share/618abbbf-6a41-45c0-bdc0-28794baa1b6c

pelican on a bicycle benchmark probably getting saturated... especially as it's become a popular way to demonstrate model ability quickly

But where is the training set of good pelicans on bikes coming from? You think they have people jigging them up internally?
Post reply on HN