Should the title here be 4.6 to 4.7 instead of the other way around?
Writing Opus 4.6 to 4.7 does make more sense for people who read left to right.
Err, how so?
Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
41–50 of 620 posts
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#42i think it is quite clear that staying with opus 4.6 is the way to go, on top of the inflation, 4.7 is quite... dumb. i think they have lobotomized this model while they were prioritizing cybersecurity and blocking people from performing potentially harmful security related tasks.
4.7 is super variable in my one day experience - it occasionally just nails a task. Then I'm back to arguing with it like it's 2023.
My experience as well, unfortunately. I am really looking forward to reading, in a few years, a proper history of the wild west years of AI scaling. What is happening in those companies at the moment must be truly fascinating. How is it possible, for instance, that I never, ever, had an instance of not being able to use Claude despite the runaway success it had, and - i'd guess - expotential increase in infra needs. When I run production workloads on vertex or bedrock i am routinely confronted with quotas, here - it always works.
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#43This, the push towards per-token API charging, and the rest are just a sign of things to come when they finally establish a moat and full monoply/duopoly, which is also what all the specialized tools like Designer and integrations are about. It's going to be a very expensive game, and the masses will be left with subpar local versions. It would be like if we reversed the democratization of compilers and coding toolin…
Yep, between this and the pricing for the code review tool that was released a couple weeks ago (15-25 a review), and the usage pricing and very expensive cost of Claude Design, I do wonder if Anthropic is making a conscious, incremental effort to raise the baseline for AI engineering tasks, especially for enterprise customers. You could call it a rug pull, but they may just be doing the math and realize this is wher…
There's been speculation that the code review might actually be Mythos. It would seem to explain the cost.
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#44AFAICT this uses a token-counting API so that it counts how many tokens are in the prompt, in two ways, so it's measuring the tokenizer change in isolation. Smarter models also sometimes produce shorter outputs and therefore fewer output tokens. That doesn't mean Opus 4.7 necessarily nets out cheaper, it might still be more expensive, but this comparison isn't really very useful.
For some real data, Artificial Analysis reported that 4.6 (max) and 4.7 (max) used 160M tokens and 100M tokens to complete their benchmark suite, respectively:
https://artificialanalysis.ai/?intelligence-efficiency=intel...
Looking at their cost breakdown, while input cost rose by $800, output cost dropped by $1400. Granted whether output offsets input will be very use-case dependent, and I imagine the delta is a lot closer at lower effort levels.
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#45[flagged]
The long-term pitch of these AI companies is that the AI will essentially replace workers for low cost.
If the models don't get to a higher level of 'intelligence' and still struggle with certain basic tasks at the SOTA while also getting more expensive, then the pitch is misleading and unlikely to happen.
So yes, I expect the price to go down.
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#46We dropped Claude. It's pretty clear this is a race to the bottom, and we don't want a hard dependency on another multi-billion dollar company just to write software
We'll be keeping an eye on open models (of which we already make good use of). I think that's the way forward. Actually it would be great if everybody would put more focus on open models, perhaps we can come up with something like the "linux/postgres/git/http/etc" of the LLMs: something we all can benefit from while it not being monopolized by a single billionarie company. Wouldn't it be nice if we don't need to pay for tokens? Paying for infra (servers, electricity) is already expensive enough
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#47Makes me think the model could actually not even be smarter necessarily, just more token dependent.
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#48My impression the reverse is true when upgrading to GPT-5.4 from GPT-5; it uses fewer tokens(?).
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#49We dropped Claude. It's pretty clear this is a race to the bottom, and we don't want a hard dependency on another multi-billion dollar company just to write software We'll be keeping an eye on open models (of which we already make good use of). I think that's the way forward. Actually it would be great if everybody would put more focus on open models, perhaps we can come up with something like the "linux/postgres/git…
Any recommendations on good open ones? What are you using primarily?
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#50The latest qwen actually performs a little better for some tasks, in my experience
latest claude still fails the car wash test