Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Opus 4.8

anthropic.com

Post reply on HN
Print view View original

871–880 of 1001 posts

Re: Claude Opus 4.8

#871

Post by arnorhs »

colonCapitalDee wrote: post #32

"Users will find Opus 4.8 to be a modest but tangible improvement on its predecessor." This is a refreshing attitude! I've also verified that you can now turn off adaptive thinking in the web UI, which is great. I've had a lot of problems with thinking not triggering and the model producing sub-par output. Glad we can finally turn it off. (I hope being able to turn off adaptive thinking is new, if I could have turned…

Yes, modest but tangible improvement - same modesty does not apply to the cost: https://artificialanalysis.ai/models/capabilities/coding#cod...

Re: Claude Opus 4.8

#873

Post by senko »

skolos wrote: post #793
senko wrote: post #385

My fav coding benchmark for frontier models is to build a simple RTS game in one file (js/html/css). Claude Code with Opus 4.8 in ultracode mode nailed it, the best result so far: https://bsky.app/profile/senko.net/post/3mmwnrkwboc2v The prompt was: Create a simple but functional real time strategy (RTS) game similar to old WarCraft, StarCraft or Command & Conquer games. The player should be able to build buildings,…

How many times did you try? Same model running multiple times can produce both very good and very bad results. In my benchmark even 10 runs often not enough to tell for sure if one model is better than another.

Usually just once (and I did just one test for this particular one), but I've found the overall quality to be relatively consistent.

There's too many confounding variables here, randomness just one of them. So I don't think of it as a definitive test (and reliable ordering), just another data point (along with actual benchmarks, pelicans, etc) to get a sense of the capabilities.

For example, I managed to get something out of DeepSeek 4 Flash quantized to 2-bit with Antirez' DwarfStar, used via Pi. Almost kinda worked! :) Which makes me optimistic for using local models for serious development soon - I'd say within a year.

Re: Claude Opus 4.8

#875

Post by bigupthewhole »

NiloCK wrote: post #77

A rambling comment: I think this is the first time we've had a third minor version bump on a frontier Anthropic model. (I count the 0.5s as major here, because they've been issued non-sequentially and also corresponded to massive capability leaps, eg, Sonnet 3.5, Opus 4.5). So now the Opus 4.5 family has successors 4.6, 4.7, and 4.8, each posting fairly modest claimed gains. My own experience w/ 4.6 and 4.7 are that…

Ive been using gpt 5.4 and 5.5 and honestly 5.4 is solving everything at the pace I need it. I'm the biggest bottle neck in terms of reviewing PRs and my own code. So having a model which can solve a complex task in 10 minutes vs 30 minutes doesn't really give me any meaningful improvement.

Also, the biggest factor is having a good planning phase. A good plan is better than even major model improvements.

Re: Claude Opus 4.8

#876

Post by orphea »

rphv wrote: post #771
jclay wrote: post #424

Earlier quoted context omitted.

It almost appears as if the code was minified. The variable names are short and formatting looks like it's written to minimize whitespace. Did it write it in this compact format all on it's own?

"Readability by humans" may no longer be as important as it once was.

Only if LLMs will start to output object code, skipping text representation.

Re: Claude Opus 4.8

#877

Post by dingdingdang »

onlyrealcuzzo wrote: post #136
NiloCK wrote: post #77

A rambling comment: I think this is the first time we've had a third minor version bump on a frontier Anthropic model. (I count the 0.5s as major here, because they've been issued non-sequentially and also corresponded to massive capability leaps, eg, Sonnet 3.5, Opus 4.5). So now the Opus 4.5 family has successors 4.6, 4.7, and 4.8, each posting fairly modest claimed gains. My own experience w/ 4.6 and 4.7 are that…

I won't be surprised if the next gen frontier models are the last. There's orders of magnitude of low hanging juice to squeeze out of smaller models. It is almost guaranteed that a 60-90B model can outperform current SOTA in coding tasks within 2-3 years (design not certain, probably unlikely). It is far less clear that a 1.2T model will be meaningfully better enough to justify training it. As far as reasoning is con…

By pointing out the exact things that will likely happen you are oddly enough hedging against (at least some of them) happening!

A) I reckon it's true that smaller models will continue to improve massively through optimization and better and better harnesses, this tech is all still very young and A LOT of resources and (good-)will is being thrown at it.

B) The 1T+ models will be able to sideload and improve upon a lot of the fundamental improvements that happen to the smaller models to speed up incredibly while getting better at tools while (on a gradient) getting -more- things right.

C) More of an observation that I think is worth keeping in mind clearly; Karl Popper's black swan and all, truth in our temporal world IS a gradient!

Re: Claude Opus 4.8

#878

Post by CamelCaseName »

tornikeo wrote: post #852
thombles wrote: post #844

Today I was a few hours into chasing down a very tricky timing-dependent bug with GPT 5.5 and we were starting to go into circles. I noticed Opus 4.8 had showed up in GitHub Copilot so I switched over and pointed it at my notes so far. Another hour of steady progress and it tracked it down to some missing synchronisation in an upstream library which was occasionally corrupting a linked list. N=1 but worth every one o…

GPT 5.5 feels worse than 5.4 for the last few weeks. Again N=1, but would be interested to see how opus 4.8 and gpt 5.4 match

You know what that means... 5.6 is dropping soon

Re: Claude Opus 4.8

#879

Post by senko »

shlewis wrote: post #701
senko wrote: post #385

My fav coding benchmark for frontier models is to build a simple RTS game in one file (js/html/css). Claude Code with Opus 4.8 in ultracode mode nailed it, the best result so far: https://bsky.app/profile/senko.net/post/3mmwnrkwboc2v The prompt was: Create a simple but functional real time strategy (RTS) game similar to old WarCraft, StarCraft or Command & Conquer games. The player should be able to build buildings,…

How much did it cost?

Token equivalent of ~ $20 (I'm on a $100 Max sub).

Re: Claude Opus 4.8

#880

Post by senko »

senko wrote: post #556
jryan49 wrote: post #514

Earlier quoted context omitted.

Kinda buggy, but impressively nonetheless. How long did it take?

It took 50 minutes, would be ~$20 in API costs (I'm on a Pro sub).

(Correction: I'm on a Max ($100/mo) sub. Realized the mistake too late, so can't edit my comment.)
Post reply on HN