Show HN: Can I run this LLM? (locally)

#1

Post by asasidh » Sat, Mar 08, 2025, 11:08 PM UTC

One of the most frequent questions one faces while running LLMs locally is: I have xx RAM and yy GPU, Can I run zz LLM model ? I have vibe coded a simple application to help you with just that.

Update: A lot of great feedback for me to improve the app. Thank you all.

Show HN: Can I run this LLM? (locally)
can-i-run-this-llm-blue.vercel.app

Re: Show HN: Can I run this LLM? (locally)

#2

Post by codingdave » Sat, Mar 08, 2025, 11:20 PM UTC

And herein lies the problem with vibe coding - accuracy is wanting.

I can absolutely run models that this site says cannot be run. Shared RAM is a thing - even with limited VRAM, shared RAM can compensate to run larger models. (Slowly, admittedly, but they work.)

Re: Show HN: Can I run this LLM? (locally)

#3

Post by asasidh » Sat, Mar 08, 2025, 11:30 PM UTC

codingdave wrote: post #2

And herein lies the problem with vibe coding - accuracy is wanting. I can absolutely run models that this site says cannot be run. Shared RAM is a thing - even with limited VRAM, shared RAM can compensate to run larger models. (Slowly, admittedly, but they work.)

yes I agree that you can run. I have personally run Ollama on a 2020 intel macbook pro. Its not a problem of vibe coding, but of the choice of logic i went with.

Re: Show HN: Can I run this LLM? (locally)

#4

Post by asasidh » Sat, Mar 08, 2025, 11:32 PM UTC

codingdave wrote: post #2

And herein lies the problem with vibe coding - accuracy is wanting. I can absolutely run models that this site says cannot be run. Shared RAM is a thing - even with limited VRAM, shared RAM can compensate to run larger models. (Slowly, admittedly, but they work.)

thanks for your feedback, there is room to show how fast or slow the model will run. I will try to update the app

Re: Show HN: Can I run this LLM? (locally)

#5

Post by fdafds » Sat, Mar 08, 2025, 11:54 PM UTC

[flagged]

Re: Show HN: Can I run this LLM? (locally)

#6

Post by asasidh » Sat, Mar 08, 2025, 11:55 PM UTC

fdafds wrote: post #5

[flagged]

no, I dont see how your post is related. maybe I am just arguing with a bot ?

Re: Show HN: Can I run this LLM? (locally)

#7

Post by ceejayoz » Sat, Mar 08, 2025, 11:58 PM UTC

asasidh wrote: post #6

fdafds wrote: post #5

[flagged]

no, I dont see how your post is related. maybe I am just arguing with a bot ?

You are. There are a couple accounts spamming this link today.

Re: Show HN: Can I run this LLM? (locally)

#8

Post by scwilbanks » Sun, Mar 09, 2025, 12:02 AM UTC

I think I would like if it also provided benchmarks. The question I have is less can I run this model, but what is the most performant (on some metric) model I can run on my current system?

Re: Show HN: Can I run this LLM? (locally)

#9

Post by drdaeman » Sun, Mar 09, 2025, 12:04 AM UTC

You forgot to define what constitutes “running”. And people have different expectations.

Re: Show HN: Can I run this LLM? (locally)

#10

Post by avereveard » Sun, Mar 09, 2025, 12:04 AM UTC

codingdave wrote: post #2

And herein lies the problem with vibe coding - accuracy is wanting. I can absolutely run models that this site says cannot be run. Shared RAM is a thing - even with limited VRAM, shared RAM can compensate to run larger models. (Slowly, admittedly, but they work.)

Also quantization and allocation strategies are a big thing for local usage. 16gb vram don't seem a lot, but you can run recent 32b model in IQ3 with their full 128k context if you allocate the kv matrix on system memory, with 15t/s and a decent prompt processing speed (just above 1000t/s on my hardware)