451–460 of 540 posts
Earlier quoted context omitted.
I ran an internal (oil and gas focused) benchmark yesterday and found Opus 4.7 was 50% cheaper than Opus 4.6, driven by significantly fewer output tokens for reasoning. It also scored 80% (vs. 60%).
That’s just adaptive reasoning, not related to the increased tokenizer costs.
Earlier quoted context omitted.
See also HP. Pretty much only Costco left.
This is where PBCs (Public Benefit Companies) and B-Corps may have a role to play. Something like that seems necessary to enable both (A) sufficient profitability to support innovation and viability in a capitalist society and (B) consideration of the public good. Traditional public companies aren't just disincentivized from caring about externalities, they're legally required to maximize shareholder profits, full st…
The "multiplier" on Github Copilot went from 3 to 7.5. Nice to see that it is actually only 20-30% and Microsoft wanting to lose money slightly slower. https://docs.github.com/fr/copilot/reference/ai-models/suppo...
Yep, and I just made a recommendation that was essentially "never enable Opus 4.7" to my org as a direct result. We have Opus 4.6 (3x) and Opus 4.5 (3x) enabled currently. They are worth it for planning . At 7.5x for 4.7, heck no. It isn't even clear it is an upgrade over Opus 4.6.
LLMs exist on a logaritmhic performance/cost frontier. It's not really clear whether Opus 4.5+ represent a level shift on this frontier or just inhabits place on that curve which delivers higher performance, but at rapidly diminishing returns to inference cost. To me, it is hard to reject this hypothesis today. The fact that Anthropic is rapidly trying to increase price may betray the fact that their recent lead is a…
The goal of every company that needs to make ever more money for investors is to earn more money while spending less. There are many ways of doing this without reducing the quality of the product, e.g. using less staff to do more, getting more compute out of same the energy, using cheaper or free energy, optimizing algorithms in ways that do not degrade quality or you grow because you gain more customers and break into new markets etc. And once you made all these optimizations and the market is saturated, then the only optimizations left are the ones where the quality goes down or the risk is increased. Quality in that sense, is what you can get away with without customers jumping ship. So you will also work on locking customers in and make jumping ship look very hard and complicated.
I find it interesting that folks are so focused on cost for AI models. Human time spent redirecting AI coding agents towards better strategies and reviewing work, remains dramatically more expensive than the token cost for AI coding, for anything other than hobby work (where you're not paying for the human labor). $200/month is an expensive hobby, but it's negligible as a business expense; SalesForce licenses cost fa…
Also, I think the $200/mo plan is subsidized by VC money and is likely hemorrhaging money for Anthropic, so it's not really meaningful to reason around that.
Earlier quoted context omitted.
That was a great promise before the models starting becoming "moody" due to their proprietors arbitrarily modifying their performance capabilities and defaults without transparency or recourse.
I still haven't seen any statistically sound data supporting that this is happening on the API (per-token pricing.) If you've got something to share I'd love to see it.
>The most striking row is user prompts: 5,608 in February vs 5,701 in March. The human put in the same effort. But the model consumed 80x more API requests and 64x more output tokens to produce demonstrably worse results.
Earlier quoted context omitted.
It's not gonna stay that way. Token cost is being massively subsidized right now. Prices will have to start increasing at some point.
You can run Qwen3 Coder today - on expensive hardware - but fairly cheaply on a token by token basis. It's no Opus, but you can get things done.
They're fun to mess around with to figure out what they can and can't do, but they're certainly not not tools in the way I can count on Codex.
Earlier quoted context omitted.
That’s just adaptive reasoning, not related to the increased tokenizer costs.
Why would I as a user be concerned about one over the other?
Unless you want to keep complaining about the model being nerfed.
Earlier quoted context omitted.
That. We already shipped 3 things this year built using Claude. The biggest one was porting two native apps into one react native app - which was originally estimated to be a 6-7 month project for a 9 FTE team, and ended up being a 2 months project with 2 people. To me, the economic value of a claude subscription used right is in the range of 10-40k eur, depending on the type of work and the developer driving it. If…
Who's going to review that output for accuracy? We'll leave performance and security as unnecessary luxuries in this age and time. In my experience, even Claude 4.6's output can't be trusted blindly it'll write flawed code and would write tests that would be testing that flawed code giving false sense of confidence and accomplishment only to be revealed upon closer inspection later. Additionally - it's age old known…
- meaningful test coverage
- internal software architecture was explicitly baked into the prompts, and we try to not go wild with vibing, but, rather, spec it well, and keep Claude on a short leash
- each feature built was followed by a round of refactoring (with Claude, but with an oversight of an opinionated human). we spend 50% building, 50% refactoring, at least. Sometimes it feels like 30/70%. Code quality matters to us, as those codebases are large and not doing this leads to very noticeable drop in Claude's perceived 'intelligence'.
- performance tests as per usual - designed by our infra engineers, not vibed
- static code analysis, and a hierarchical system of guardrails (small claude.md + lots of files referenced there for various purposes). Not quite fond of how that works, Claude has been always very keen to ignore instructions and go his own way (see: "short leash, refactor often").
- pentests with regular human beings
The one project I mentioned - 2 months for a complete rewrite - was about a week of working on the code and almost 2 months spent on reviews, tests, and of course some of that time was wasted as we were doing this for the first time for such a large codebase. The rewritten app is doing fine in production for a while now.
I can only compare the outputs to the quality of the outputs of our regular engineering teams. It compares fine vs. good dev teams, IMHO.
LLMs exist on a logaritmhic performance/cost frontier. It's not really clear whether Opus 4.5+ represent a level shift on this frontier or just inhabits place on that curve which delivers higher performance, but at rapidly diminishing returns to inference cost. To me, it is hard to reject this hypothesis today. The fact that Anthropic is rapidly trying to increase price may betray the fact that their recent lead is a…