Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Opus 4.5

anthropic.com

Post reply on HN
Print view View original

501–510 of 525 posts

Re: Claude Opus 4.5

#501

Post by Wowfunhappy »

zsoltkacsandi wrote: post #197
lukev wrote: post #154

Earlier quoted context omitted.

There are two possible explanations for this behavior: the model nerf is real, or there's a perceptual/psychological shift. However, benchmarks exist. And I haven't seen any empirical evidence that the performance of a given model version grows worse over time on benchmarks (in general.) Therefore, some combination of two things are true: 1. The nerf is psychologial, not actual. 2. The nerf is real but in a way that…

> The nerf is psychologial, not actual Once I tested this, I gave the same task for a model after the release and a couple weeks later. In the first attempt it produced a well-written code that worked beautifully, I started to worry about the jobs of the software engineers. Second attempt was a nightmare, like a butcher acting as a junior developer performing a surgery on a horse. Is this empirical evidence? And this…

I think this is pretty easy to explain psychologically.

The first time you see a dog that can make pancakes, you’re really focused on the fact that a dog is making pancakes.

After a few weeks of having them for breakfast, you start to notice that the pancakes are actually kind of overcooked and don’t taste that good. Sure it’s impressive that a dog made them, but what use are sub-par pancakes? You’re naturally more focused on what it can’t do than what it can.

Re: Claude Opus 4.5

#502

Post by cyrusradfar »

irthomasthomas wrote: post #122
cyrusradfar wrote: post #77

I'm curious if others are finding that there's a comfort in staying within the Claude ecosystem because when it makes a mistake, we get used to spotting the pattern. I'm finding that when I try new models, their "stupid" moments are more surprising and infuriating. Given this tech is new, the experience of how we relate to their mistakes is something I think a bit about. Am I alone here, are others finding themselves…

I guess you where not around a few months back when they over-optimized and served a degraded model for weeks.

that's funny you just made me connect the dots.

I was! I spent several days spinning in place after I thought it could help me clean up my code quality with biome. Afterwards it destroyed the whole app and I needed to figure out how it worked -- that need, inspired me to prototype and extension for vccode I'm actually still building :)

Re: Claude Opus 4.5

#503

Post by andreybaskov »

threeducks wrote: post #217
andreybaskov wrote: post #112

Does anyone know or have a guess on the size of this latest thinking models and what hardware they use to run inference? As in how much memory and what quantization it uses and if it's "theoretically" possible to run it on something like Mac Studio M3 Ultra with 512GB RAM. Just curious from theoretical perspective.

Rough ballpark estimate: - Amazon Bedrock serves Claude Opus 4.5 at 57.37 tokens per second: https://openrouter.ai/anthropic/claude-opus-4.5 - Amazon Bedrock serves gpt-oss-120b at 1748 tokens per second: https://openrouter.ai/openai/gpt-oss-120b - gpt-oss-120b has 5.1B active parameters at approximately 4 bits per parameter: https://huggingface.co/openai/gpt-oss-120b To generate one token, all active parameters must…

Thanks! That's a great way to analyze it by comparing to open source models. Though I wonder if they use the same hardware for gpt-oss-120b and Claude Opus.

Re: Claude Opus 4.5

#504

Post by camdenreslink »

lukev wrote: post #154
unsupp0rted wrote: post #46

This is gonna be game-changing for the next 2-4 weeks before they nerf the model. Then for the next 2-3 months people complaining about the degradation will be labeled “skill issue”. Then a sacrificial Anthropic engineer will “discover” a couple obscure bugs that “in some cases” might have lead to less than optimal performance. Still largely a user skill issue though. Then a couple months later they’ll release Opus 4…

There are two possible explanations for this behavior: the model nerf is real, or there's a perceptual/psychological shift. However, benchmarks exist. And I haven't seen any empirical evidence that the performance of a given model version grows worse over time on benchmarks (in general.) Therefore, some combination of two things are true: 1. The nerf is psychologial, not actual. 2. The nerf is real but in a way that…

As a personal anecdote, I had a fairly involved application that built up a context with a lot of custom prompting and created a ~1000 word output. I could run my application over and over again to inspect the results. It was fairly reproducible.

I was having really nice results with the o4-mini model with high thinking. A little while after GPT-5 came out I revisited my application and tried to continue. The o4-mini results were unusable, while the GPT-5 results were similar to what I had before. I'm not sure what happened to the model in those ~4-5 months I set it down, but there was real degradation.

Re: Claude Opus 4.5

#505

Post by irthomasthomas »

cyrusradfar wrote: post #502
irthomasthomas wrote: post #122

Earlier quoted context omitted.

I guess you where not around a few months back when they over-optimized and served a degraded model for weeks.

that's funny you just made me connect the dots. I was! I spent several days spinning in place after I thought it could help me clean up my code quality with biome. Afterwards it destroyed the whole app and I needed to figure out how it worked -- that need, inspired me to prototype and extension for vccode I'm actually still building :)

Yep, that was it! That really turned me off anthropic and closed models until they provide regular quality tests. I use chutes ai, now. They tell you exactly which model/quant and server config they use, so you know if you have trouble with a task, it's not the model.

Re: Claude Opus 4.5

#507

Post by lymbo »

thelittleone wrote: post #367
Aeolun wrote: post #297

Earlier quoted context omitted.

It’s important to note that with the introduction of Sonnet 4.5 they absolutely cratered the limits, and the opus limits in specific, so this just sort of comes closer to the situation we were actually in before.

That's probably true, but whereas before I hit max 200. Limits once a week or so. Now I have multiple projects running 16hrs a day some with 3-4 worktrees, and haven't hit limits for several weeks.

Holy smokes, are you willing to share any vague details of what you’re running for 16 hours per day?

Re: Claude Opus 4.5

#508

Post by riku_iki »

energy123 wrote: post #338
ddxv wrote: post #328

The LLMs rate of improvement has really slowed down. This looks like a minor improvement in terms of accuracy and big gains from efficiency.

14 months ago we had GPT-4 and now we have models that can get a gold medal at the IMO. But sure, if you curve fit to the last 3 months you could say things are slowing down, but that's hyper fixating on a very small amount of information.

it could be results of corps focusing resources on IMO in PR wars, and results is not as generalizable outside this niche.

Re: Claude Opus 4.5

#509

Post by tiahura »

easygenes wrote: post #110
config_yml wrote: post #53

Earlier quoted context omitted.

I use plan mode in claude code, then use gpt-5 in codex to review the plan and identify gaps and feed it back to claude. Results are amazing.

Yeah, I’ve used vatiations of the “get frontier models to cross-check and refine each others work” pattern for years now and it really is the path to the best outcomes in situations where you would otherwise hit a wall or miss important details.

It’s my approach in legal as well. Claude formulates its draft, then it prompts codex and gemini for theirs. Claude then makes recommendations for edits to its draft based on others. Gemini’s plan is almost always the worst, but even it frequently has at least one good point to make.

Re: Claude Opus 4.5

#510

Post by tiahura »

danielbln wrote: post #201
config_yml wrote: post #53

Earlier quoted context omitted.

I use plan mode in claude code, then use gpt-5 in codex to review the plan and identify gaps and feed it back to claude. Results are amazing.

If you're not already doing that you can wire up a subagent that invokes codex in non interactive mode. Very handy, I run Gemini-cli and codex subagents in parallel to validate plans or implementations.

This is the way. However, there a a lot of approaches to ensemble approaches. I wish there were some good benchmarks for various domains.
Post reply on HN