Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Opus 4.5

anthropic.com

Post reply on HN
Print view View original

361–370 of 525 posts

Re: Claude Opus 4.5

#361

Post by [deleted] »

unsupp0rted wrote: post #344
roywiggins wrote: post #339

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Regression_toward_the_mean The way this works is: 1) x% of users have an exceptional first experience by chance. Nobody who has a meh first experience bothers to try a second time. 2) x²% of users also have an exceptional second experience by chance 3) So a lot of people with a great first experience think the model started off great and got suddenly worse Suppose it's 25% that have a re…

If by "second" and "third" experience you mean "after 2 ~ 4 weeks of all-day usage"

[deleted]

Re: Claude Opus 4.5

#362

Post by tkgally »

simonw wrote: post #100

Notes and two pelicans: https://simonwillison.net/2025/Nov/24/claude-opus/

I added Opus 4.5 to my benchmark of 30 alternatives to your now-classic pelican-bicycle prompt (e.g., “Generate an SVG of a dragonfly balancing a chandelier”). Nine models are now represented:

https://gally.net/temp/20251107pelican-alternatives/index.ht...

Re: Claude Opus 4.5

#363

Post by windexh8er »

AtNightWeCode wrote: post #261
llamasushi wrote: post #30

The burying of the lede here is insane. $5/$25 per MTok is a 3x price drop from Opus 4. At that price point, Opus stops being "the model you use for important things" and becomes actually viable for production workloads. Also notable: they're claiming SOTA prompt injection resistance. The industry has largely given up on solving this problem through training alone, so if the numbers in the system card hold up under a…

The cost of tokens in the docs is pretty much a worthless metric for these models. Only way to go is to plug it in and test it. My experience is that Claude is an expert at wasting tokens on nonsense. Easily 5x up on output tokens comparing to ChatGPT and then consider that Claude waste about 2-3x of tokens more by default.

This is spot on. The amount of wasteful output tokens from Claude is crazy. The actual output you're looking for might be better, but you're definitely going to pay for it in the long run.

The other angle here is that it's very easy to waste a ton of time and tokens with cheap models. Or you can more slowly dig yourself a hole with the SOTA models. But either way, and even with 1M tokens of context - things spiral at some point. It's just a question of whether you can get off the tracks with a working widget. It's always frustrating to know that "resetting" the environment is just handing over some free tokens to [model-provider-here] to recontextualize itself. I feel like it's the ultimate Office Space hack, likely unintentional, but really helps drive home the point of how unreliable all these offerings are.

Re: Claude Opus 4.5

#364

Post by jsnell »

nostrademons wrote: post #347
jsnell wrote: post #336

Earlier quoted context omitted.

Nobody subsidizes LLM APIs. There is a reason to subsidize free consumer offerings: those users are very sticky, and won't switch unless the alternative is much better. There might be a reason to subsidize subscriptions, but only if your value is in the app rather than the model. But for API use, the models are easily substituted, so market share is fleeting. The LLM interface being unstructured plain text makes it s…

LLM APIs are more sticky than many other computing APIs. Much of the eng work is in the prompt engineering, and the prompt engineering is pretty specific to the particular LLM you're using. If you randomly swap out the API calls, you'll find you get significantly worse results, because you tuned your prompts to the particular LLM you were using. It's much more akin to a programming language or platform than a typical…

No, LLMs really are not more sticky than traditional APIs. Normal APIs are unforgiving in their inputs and rigid in their outputs. No matter how hard you try, Hyrum's Law will get you over and over again. Every migration is an exercise in pain. LLMs are the ultimate adapting, malleable tool. It doesn't matter if you'd carefully tuned your prompt against a specific six months old model. The new model of today is sufficiently smarter that it'll do a better job despite not having been tuned on those specific prompts.

This isn't even theory, we can observe the swings in practice on Openrouter.

If the value was in prompt engineering, people would stick to specific old versions of models, because a new version of a given model might as well be a totally different model. It will behave differently, and will need to be qualified again. But of course only few people stick with the obsolete models. How many applications do you think still use a model released a year ago?

Re: Claude Opus 4.5

#365

Post by ddxv »

energy123 wrote: post #338
ddxv wrote: post #328

The LLMs rate of improvement has really slowed down. This looks like a minor improvement in terms of accuracy and big gains from efficiency.

14 months ago we had GPT-4 and now we have models that can get a gold medal at the IMO. But sure, if you curve fit to the last 3 months you could say things are slowing down, but that's hyper fixating on a very small amount of information.

Yes, that is what I'm saying, that 14 months ago the rate of change was noticeably faster. Lately the new models are much less groundbreaking and increasing in the volume of output and decreasing in cost.

Re: Claude Opus 4.5

#366

Post by [deleted] »

ewoodrich wrote: post #252
baq wrote: post #248

Earlier quoted context omitted.

This isn’t how you should be benchmarking models. You should give it the same task n times and see how often it succeeds and/or how long it takes to be successful (see also the 50% time horizon metric by METR).

I was pretty disappointed to learn that the METR metric isn't actually evaluating a model's ability to complete long duration tasks. They're using the estimated time a human would take on a given task. But it did explain my increasing bafflement at how the METR line keeps steadily going up despite my personal experience coding daily with LLMs where they still frequently struggle to work independently for 10 minutes w…

[deleted]

Re: Claude Opus 4.5

#367

Post by thelittleone »

Aeolun wrote: post #297
tekacs wrote: post #60

Earlier quoted context omitted.

This is also super relevant for everyone who had ditched Claude Code due to limits: > For Claude and Claude Code users with access to Opus 4.5, we’ve removed Opus-specific caps. For Max and Team Premium users, we’ve increased overall usage limits, meaning you’ll have roughly the same number of Opus tokens as you previously had with Sonnet. We’re updating usage limits to make sure you’re able to use Opus 4.5 for daily…

It’s important to note that with the introduction of Sonnet 4.5 they absolutely cratered the limits, and the opus limits in specific, so this just sort of comes closer to the situation we were actually in before.

That's probably true, but whereas before I hit max 200. Limits once a week or so. Now I have multiple projects running 16hrs a day some with 3-4 worktrees, and haven't hit limits for several weeks.

Re: Claude Opus 4.5

#368

Post by Aeolun »

Wowfunhappy wrote: post #318
Aeolun wrote: post #299

Earlier quoted context omitted.

THIS, FOR EXAMPLE. IT IS MUCH MORE REPRESENTATIVE OF HOW ANNOYING IT IS TO READ THAN A SINGLE CAPITALIZATION OF that.

But again, if that is what the guideline is referring to, why does it say "If you want to emphasize a _word or phrase_". By my reading, it is quite explicitly including single words!

I’m saying that being pedantic on HN is a worse sin than capitalizing a single word. Being technically correct isn’t really relevant to how annoying people think you are being.

Re: Claude Opus 4.5

#369

Post by thelittleone »

sqs wrote: post #287
tmaly wrote: post #267

Earlier quoted context omitted.

what is the typical usage pattern that would result in these cost figures?

Using small threads (see https://ampcode.com/@sqs for some of my public threads). If you use very long threads and treat it as a long-and-winding conversation, you will get worse results and pay a lot more.

The context usage awareness is a bit boost for this in my experience. I use speckit and have setup to wrap up tasks when at least 20% of context remaining with a summary of progress, followed by /clear, insert summary and continue. This has reduced compacts almost entirely.

Re: Claude Opus 4.5

#370

Post by highfrequency »

jsnell wrote: post #336
nostrademons wrote: post #163

Earlier quoted context omitted.

Why? They just closed a $13B funding round. Entirely possible that they're selling below-cost to gain marketshare; on their current usage the cloud computing costs shouldn't be too bad, while the benefits of showing continued growth on their frontier models is great. Hell, for all we know they may have priced Opus 4.1 above cost to show positive unit economics to investors, and then drop the price of Opus 4.5 to spur…

Nobody subsidizes LLM APIs. There is a reason to subsidize free consumer offerings: those users are very sticky, and won't switch unless the alternative is much better. There might be a reason to subsidize subscriptions, but only if your value is in the app rather than the model. But for API use, the models are easily substituted, so market share is fleeting. The LLM interface being unstructured plain text makes it s…

> But for API use, the models are easily substituted, so market share is fleeting. The LLM interface being unstructured plain text makes it simpler to upgrade to a smarter model than than it used to be to swap a library or upgrade to a new version of the JVM.

Agree that the plain text interface (which enables extremely fast user adoption) also makes the product less sticky. I wonder if this is part of the incentive to push for specialized tool calling interfaces / MCP stuff - to engineer more lock in by increasing the model specific surface area.

Post reply on HN