Looking for the cheapest place to deploy llama 3.1 model? Don't worry we have found it so you don't have to.
Show HN: Open-source LLM provider price comparison
github.com
Show HN: Open-source LLM provider price comparison
1–10 of 34 posts
Re: Show HN: Open-source LLM provider price comparison
#2nice, literally what i need on my recent project that roasts HN profiles.
how about adding more models and providers?
and making a sortable table?
Re: Show HN: Open-source LLM provider price comparison
#3Litellm maintains list of model infos here:
https://github.com/BerriAI/litellm/blob/main/model_prices_an...
Simple UI to search:
Re: Show HN: Open-source LLM provider price comparison
#4Nice I built something similar
https://huggingface.co/spaces/Whiteshadow12/llm-pricing-calc...
I like your charting, many have taken this task and then lose interest.
similar other tools for inspiration https://llmprices.dev/ https://www.llmpricing.app/
What no one is doing is focusing on GPUs, what is the cost of running L3-8B on an A100 or H100 per second.
Re: Show HN: Open-source LLM provider price comparison
#5The problem with these Open weights LLMs hosted by these provider is that we don't what's the precision of the LLM, that makes a huge difference in the speed and cost (compute).
I think Together recently introduced a different price tier based on precision but otherwise it is usually dark.
Re: Show HN: Open-source LLM provider price comparison
#6The problem with these Open weights LLMs hosted by these provider is that we don't what's the precision of the LLM, that makes a huge difference in the speed and cost (compute). I think Together recently introduced a different price tier based on precision but otherwise it is usually dark.
Something Open Router get's close to that https://openrouter.ai/models/meta-llama/llama-3.1-405b-instr...
Re: Show HN: Open-source LLM provider price comparison
#7Nice I built something similar https://huggingface.co/spaces/Whiteshadow12/llm-pricing-calc... I like your charting, many have taken this task and then lose interest. similar other tools for inspiration https://llmprices.dev/ https://www.llmpricing.app/ What no one is doing is focusing on GPUs, what is the cost of running L3-8B on an A100 or H100 per second.
Yeah we want to do exactly this, benchmark and add more data from differnt gpus/cloud providers, will appreciate your help a lot!
There are many inference engines which can be tested and updated to find best inference methods
Re: Show HN: Open-source LLM provider price comparison
#8Litellm maintains list of model infos here: https://github.com/BerriAI/litellm/blob/main/model_prices_an... Simple UI to search: https://models.litellm.ai/?q=llama3
Thank you! yeah great source.
Do they track throughput for open source models? and inference engines
Thats kind of data I want to find as well
Re: Show HN: Open-source LLM provider price comparison
#9nice, literally what i need on my recent project that roasts HN profiles. how about adding more models and providers? and making a sortable table?
Honestly this is very very fresh, I was tinkering with hosting some models and wanted to optimize costs, tried few inference engines. Just want to collaborate on organizing data.
Agree, we will add a MUI table very soon. Also some charts.
I genuinely want someone to roast the way I did my benchmark process described there. Want something good enough yet easy to run.
Re: Show HN: Open-source LLM provider price comparison
#10The problem with these Open weights LLMs hosted by these provider is that we don't what's the precision of the LLM, that makes a huge difference in the speed and cost (compute). I think Together recently introduced a different price tier based on precision but otherwise it is usually dark.
Exactly, always best to rely on your hardware, we need to collect/add more data from self hosted models on different gpus/clouds to compare