11–20 of 79 posts
Maybe a small triage AI to decide how effectively models handle certain prompts to preserve spending for the difficult tasks.
Does anything like this exist yet?
I don't understand how the Claude functionality works. As far as I know Anthropic haven't released the tokenizer for Claude - unlike OpenAI's tiktoken - but your tool lists the Claude 3 models as supported. How are you counting tokens for those?
At this moment, Tokencost uses the OpenAI tokenizer as a default tokenizer, but this would be a welcome PR!
Nice. Any plans to add calculations for image input for the models that allow that?
I dig it! Kind of related, but I made a comparison of LLM API costs vs their leaderboard performance to gauge which models can be more bang for the buck [0] [0] https://llmcompare.net
I don't understand how the Claude functionality works. As far as I know Anthropic haven't released the tokenizer for Claude - unlike OpenAI's tiktoken - but your tool lists the Claude 3 models as supported. How are you counting tokens for those?
Anthropic actually has a Claude 3 tokenizer tucked away in one of their repos: https://github.com/anthropics/anthropic-tokenizer-typescript At this moment, Tokencost uses the OpenAI tokenizer as a default tokenizer, but this would be a welcome PR!
I've been bugging Anthropic about this for a while, they said that releasing a new tokenizer is not on their current roadmap.
Earlier quoted context omitted.
Anthropic actually has a Claude 3 tokenizer tucked away in one of their repos: https://github.com/anthropics/anthropic-tokenizer-typescript At this moment, Tokencost uses the OpenAI tokenizer as a default tokenizer, but this would be a welcome PR!
"This package can be used to count tokens for Anthropic's older models. As of the Claude 3 models, this algorithm is no longer accurate [...]" I've been bugging Anthropic about this for a while, they said that releasing a new tokenizer is not on their current roadmap.
Earlier quoted context omitted.
"This package can be used to count tokens for Anthropic's older models. As of the Claude 3 models, this algorithm is no longer accurate [...]" I've been bugging Anthropic about this for a while, they said that releasing a new tokenizer is not on their current roadmap.
Imagine a coffee shop refusing to have a price list until after the coffee's been made.
Frequently, contracts will have room for additional charges if circumstances change even a little, or products will have a market rate (fish, equity, etc.).
It might seem absurd but variable cost things are not uncommon.
Earlier quoted context omitted.
Imagine a coffee shop refusing to have a price list until after the coffee's been made.
In many countries a taxi won't tell you how much the ride will cost. The first time I traveled to somewhere that negotiated the cost up front it blew my mind. Frequently, contracts will have room for additional charges if circumstances change even a little, or products will have a market rate (fish, equity, etc.). It might seem absurd but variable cost things are not uncommon.