41–50 of 79 posts
Earlier quoted context omitted.
"This package can be used to count tokens for Anthropic's older models. As of the Claude 3 models, this algorithm is no longer accurate [...]" I've been bugging Anthropic about this for a while, they said that releasing a new tokenizer is not on their current roadmap.
Imagine a coffee shop refusing to have a price list until after the coffee's been made.
With all the options there seems like an opportunity for a single point API that can take a series of prompts, a budget and a quality hint to distribute batches for most bang for buck. Maybe a small triage AI to decide how effectively models handle certain prompts to preserve spending for the difficult tasks. Does anything like this exist yet?
With all the options there seems like an opportunity for a single point API that can take a series of prompts, a budget and a quality hint to distribute batches for most bang for buck. Maybe a small triage AI to decide how effectively models handle certain prompts to preserve spending for the difficult tasks. Does anything like this exist yet?
In that case you might as well make ~20 API calls to each LLM under consideration, and evaluate the results yourself.
It's far easier to evaluate a model's performance on a given prompt by looking at the output than by looking at the input alone.
Earlier quoted context omitted.
Imagine a coffee shop refusing to have a price list until after the coffee's been made.
In many countries a taxi won't tell you how much the ride will cost. The first time I traveled to somewhere that negotiated the cost up front it blew my mind. Frequently, contracts will have room for additional charges if circumstances change even a little, or products will have a market rate (fish, equity, etc.). It might seem absurd but variable cost things are not uncommon.
I've only ever seen: fixed price based on destination (typically for fares originating from an airport), negotiated, or metered. A better analog analogy would be metered pricing, but where the cost per mile is a secret.
What a nice product! I think the way to count tokens depends on the language, but is this only supported in English?
Here's actual output from the GPT-4o tokenizer for English and Hindi:
>>> [enc.decode([x]) for x in enc.encode("proselytization")]
['pros', 'ely', 't', 'ization']
>>> [enc.decode([x]) for x in enc.encode("पर्यावरणवाद")]
['पर', '्य', 'ावरण', 'वाद']Earlier quoted context omitted.
$/million tokens is the standard pricing metric.
standard ≠ good
Earlier quoted context omitted.
"This package can be used to count tokens for Anthropic's older models. As of the Claude 3 models, this algorithm is no longer accurate [...]" I've been bugging Anthropic about this for a while, they said that releasing a new tokenizer is not on their current roadmap.
Imagine a coffee shop refusing to have a price list until after the coffee's been made.