Very cool! Is this cost directory you're using the best source for historical cost per 1M tokens? https://github.com/BerriAI/litellm/blob/main/model_prices_an...
Show HN: Token price calculator for 400+ LLMs
21–30 of 79 posts
Re: Show HN: Token price calculator for 400+ LLMs
#22With all the options there seems like an opportunity for a single point API that can take a series of prompts, a budget and a quality hint to distribute batches for most bang for buck. Maybe a small triage AI to decide how effectively models handle certain prompts to preserve spending for the difficult tasks. Does anything like this exist yet?
I have yet to find a use case where quality can be traded off.
Would love to hear what you had in mind.
Re: Show HN: Token price calculator for 400+ LLMs
#23Very cool! Is this cost directory you're using the best source for historical cost per 1M tokens? https://github.com/BerriAI/litellm/blob/main/model_prices_an...
Best one I've found out there. If there's another, very open to replacing
Re: Show HN: Token price calculator for 400+ LLMs
#24Re: Show HN: Token price calculator for 400+ LLMs
#25With all the options there seems like an opportunity for a single point API that can take a series of prompts, a budget and a quality hint to distribute batches for most bang for buck. Maybe a small triage AI to decide how effectively models handle certain prompts to preserve spending for the difficult tasks. Does anything like this exist yet?
I have yet to find a use case where quality can be traded off. Would love to hear what you had in mind.
Every single use case of LLMs inherently sacrifices quality, whether the developers are willing to admit it or not. I agree with you though that there aren't many use cases where end users would knowingly accept the trade off.
Re: Show HN: Token price calculator for 400+ LLMs
#26With all the options there seems like an opportunity for a single point API that can take a series of prompts, a budget and a quality hint to distribute batches for most bang for buck. Maybe a small triage AI to decide how effectively models handle certain prompts to preserve spending for the difficult tasks. Does anything like this exist yet?
I have yet to find a use case where quality can be traded off. Would love to hear what you had in mind.
It is not so much a drop in quality as there are tasks that every model above a certain threshold will perform equally.
Most can do 2+2 = 4.
One test prompt I use on LLMs is asking it to produce a JavaScript function that takes an ImageData object and returns a new ImageData object with an all direction Sobel edge detection. Quite a lot of even quite small models can generate functions like this.
In general, I don't even think this is a question that needs to be answered. A lot of API providers have different quality/price tiers. The fact that people are using the different tiers should be sufficient to show that at least some people are finding cases where cheaper models are good enough.
Re: Show HN: Token price calculator for 400+ LLMs
#27Re: Show HN: Token price calculator for 400+ LLMs
#28Re: Show HN: Token price calculator for 400+ LLMs
#29[flagged]
Re: Show HN: Token price calculator for 400+ LLMs
#30I dig it! Kind of related, but I made a comparison of LLM API costs vs their leaderboard performance to gauge which models can be more bang for the buck [0] [0] https://llmcompare.net
Nice, can you make a triangle with (cost, performance, speed)? That would show the tradeoffs.