If I want to run LLMs model like Qwen or gpt locally how much would it cost me and I want to connect it to my main website creating an API link, also which models would be best
How much cost would it take to run a LLMs locally
1–7 of 7 posts
Re: How much cost would it take to run a LLMs locally
#2First, check what LLMs your system can actually handle based on your RAM/VRAM, then choose the model variant accordingly.
If you want to use the model through an API for your website, a good setup would be a dedicated machine/server to host it, since running LLMs locally can consume a lot of memory.
Depending on your hardware, you can look at smaller variants of Gemma or Qwen Coder.
Another option is to use a hosted inference provider. It’ll cost a little, but you’ll usually get much faster inference compared to running locally especially if your system isn’t high-end.
Re: How much cost would it take to run a LLMs locally
#3go to https://huggingface.co/settings/hardware?next=%2Fmodels and add your hardware, it will show models you can run on it
Re: How much cost would it take to run a LLMs locally
#4First, check what LLMs your system can actually handle based on your RAM/VRAM, then choose the model variant accordingly. If you want to use the model through an API for your website, a good setup would be a dedicated machine/server to host it, since running LLMs locally can consume a lot of memory. Depending on your hardware, you can look at smaller variants of Gemma or Qwen Coder. Another option is to use a hosted…
What should be the minimum hardware requirements I got Nvidia processor with a high quality graphic card and 16 gb ram
Re: How much cost would it take to run a LLMs locally
#5Try TurboLLM (https://turbollm.dev/).
Re: How much cost would it take to run a LLMs locally
#6[flagged]
Re: How much cost would it take to run a LLMs locally
#7[flagged]