The real prices of frontier models
playcode.io
The real prices of frontier models
1–10 of 91 posts
Re: The real prices of frontier models
#2Tokenzier aside, a report shared on reddit found that the GPT 5.6 (edit: 5.5) series are incredibly thrifty with CoTs, resulting in cheaper bills than GLM 5.2 (let alone Opus/Fable): https://www.reddit.com/r/ZaiGLM/s/rUoG5adkPh
Chattiness remains an open issue for some of the SoTA open weights & (to a lesser extent) Claude.
Re: The real prices of frontier models
#3The fact that OpenAI documents theirs is already a big improvement over Anthropic. But, also, the OpenAI tokenizer got more efficient when they last updated it, rather than less. https://mdstudio.app/o200k-base-tokenizer
Re: The real prices of frontier models
#4This piece focuses on the cost differences from the tokenizer, which do matter, but I wish they emphasized more that even adding the tokenizer to your calculation doesn't provide you with a good way to calculate cost for agentic coding tasks.
Other traits where models differ that have an even greater impact on your total spend:
* How much context do they load in to solve a given task?
* How long do they spend thinking to get equivalent results?
* How many times do they stop and ask you for input, and are you there to respond to them before the cache runs out?
* Etc.
Incorporating the tokenizer just makes a very imprecise measurement of cost a little bit more precise, but in my own experience I have not found that the token cost is a significant driver of task cost whether or not you incorporate the tokenizer. Everything else about the model's behavior has a much larger impact.
Re: The real prices of frontier models
#5Are there any advantages of the new tokenizer? Does it have a larger or smaller vocabulary or just differently weighted?
Re: The real prices of frontier models
#6Tokenzier aside, a report shared on reddit found that the GPT 5.6 (edit: 5.5) series are incredibly thrifty with CoTs, resulting in cheaper bills than GLM 5.2 (let alone Opus/Fable): https://www.reddit.com/r/ZaiGLM/s/rUoG5adkPh Chattiness remains an open issue for some of the SoTA open weights & (to a lesser extent) Claude.
That link does not mention 5.6.
Re: The real prices of frontier models
#7My take from this is that Anthropic is screwing us again. I hope AMD shows them up again.
Re: The real prices of frontier models
#8Yeah, Anthropic's current tokenizer in Sonnet 5/Opus 4.8/Fable 5 is much worse than OpenAI's. Also, OpenAI has been using their current o200k_base from the day GPT-4o came out over two years ago. Just a few of my own tests:
- A ~2000-2002 legacy C++ game codebase at about ~90kloc: GPT 1.12M, Claude 2.2M
- A ~30kloc TypeScript codebase: GPT 260K, Claude 437K
In the end, GPT's current tokenizer is ~1.6x-2x better than Claude's current one, depending on your data. And you can check for free for both, for OpenAI just use the open-source libraries, for Anthropic - you have to use their count_tokens endpoint as they don't publish the tokenizer, but the endpoint is free (and allows requests over 1M tokens as well).
Re: The real prices of frontier models
#9this is surprisingly high delta. to make matters worse, reasoning tokens account for the majority of tokens and they are completely opaque so it's hard to tell how much of that is prose or code
Re: The real prices of frontier models
#10The real price is how often a model uses subagents that scan your whole repository multiplied by thinking level.