Does anyone know what changed in the tokenizer? Does it output multiple tokens for things that were previously one token?
Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
11–20 of 620 posts
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#12[deleted]
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#13i think it is quite clear that staying with opus 4.6 is the way to go, on top of the inflation, 4.7 is quite... dumb. i think they have lobotomized this model while they were prioritizing cybersecurity and blocking people from performing potentially harmful security related tasks.
4.7 is super variable in my one day experience - it occasionally just nails a task. Then I'm back to arguing with it like it's 2023.
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#14Wow this is pretty spectacular. And with the losses anthro and OAI are running, don’t expect this trend to change. You will get incremental output improvements for a dramatically more expensive subscription plan.
Indeed, and if we accept the argument of this tech approaching AGI, we should expect that within x years, the subscription cost may exceed the salary cost of a junior dev.
To be clear, I'm not saying that it's a good thing, but it does seem to be going in this direction.
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#15Wow this is pretty spectacular. And with the losses anthro and OAI are running, don’t expect this trend to change. You will get incremental output improvements for a dramatically more expensive subscription plan.
[deleted]
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#16is it really unthinkable that another oss/local model will be released by deepseek, alibaba, or even meta that once again give these companies a run for their money
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#17This, the push towards per-token API charging, and the rest are just a sign of things to come when they finally establish a moat and full monoply/duopoly, which is also what all the specialized tools like Designer and integrations are about. It's going to be a very expensive game, and the masses will be left with subpar local versions. It would be like if we reversed the democratization of compilers and coding toolin…
Yep, between this and the pricing for the code review tool that was released a couple weeks ago (15-25 a review), and the usage pricing and very expensive cost of Claude Design, I do wonder if Anthropic is making a conscious, incremental effort to raise the baseline for AI engineering tasks, especially for enterprise customers.
You could call it a rug pull, but they may just be doing the math and realize this is where pricing needs to shift to before going public.
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#18This, the push towards per-token API charging, and the rest are just a sign of things to come when they finally establish a moat and full monoply/duopoly, which is also what all the specialized tools like Designer and integrations are about. It's going to be a very expensive game, and the masses will be left with subpar local versions. It would be like if we reversed the democratization of compilers and coding toolin…
If only there were an Open AI company who's mandate, built into the structure of the company, were to make frontier models available to everyone for the good of humanity.
Oh well
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#19AFAICT this uses a token-counting API so that it counts how many tokens are in the prompt, in two ways, so it's measuring the tokenizer change in isolation. Smarter models also sometimes produce shorter outputs and therefore fewer output tokens. That doesn't mean Opus 4.7 necessarily nets out cheaper, it might still be more expensive, but this comparison isn't really very useful.
Re: Anonymous request-token comparisons from Opus 4.6 and Opus 4.7
#20I'm going to suggest what's going on here is Hanlon's Razor for models: "Never attribute to malice that which is adequately explained by a model's stupidity."
In my opinion, we've reached some ceiling where more tokens lead only to incremental improvements. A conspiracy seems unlikely given all providers are still competing for customers and a 50% token drives infra costs up dramatically too.