Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Find the best local LLM for your hardware, ranked by benchmarks

github.com

Post reply on HN
Print view View original

71–80 of 80 posts

Re: Show HN: Find the best local LLM for your hardware, ranked by benchmarks

#71

Post by bityard »

hazelnut wrote: post #56
Aurornis wrote: post #43

1. The results of this tool are not good. It’s recommending outdated models like Qwen2.5 series and missing good new models. 2. This could have been a single web page that runs in your browser and lets you enter hardware specs, like all of the other tools like this. It is not a good idea to install and run unknown projects like this on your computer in this age. 3. The project is very obviously vibecoded, down to the…

There is apparently a marketing.md file that was deleted 25min ago with the strategy to post on HN. https://github.com/Andyyyy64/whichllm/commit/2cefaea1cc5d2de... I think your hunch is very much spot on. It doesn’t look trustworthy at all.

Man. Every time I see a Reddit or HN post starting with "I got tired of..." I already know that it's going to be self-promoted AI slop.

Re: Show HN: Find the best local LLM for your hardware, ranked by benchmarks

#72

Post by rafram »

embedding-shape wrote: post #15
jordiburgos wrote: post #10

This is very helpful too: https://www.canirun.ai/

Love that it defaults to the GPU being "NVIDIA GeForce 8800 GTX", a GPU released in 2006 with ~700MB of VRAM... The estimates seems far off as well, took https://www.canirun.ai/model/gpt-oss-120b as an example, with a RTX Pro 6000 and every single number is off, and notably misses estimation for the most important quant for GPT-OSS, the MXFP4 variant.

The default for me was M1. I think it tries to guess using WebGPU.

Re: Show HN: Find the best local LLM for your hardware, ranked by benchmarks

#73

Post by aggregator-ios »

Aurornis wrote: post #43

1. The results of this tool are not good. It’s recommending outdated models like Qwen2.5 series and missing good new models. 2. This could have been a single web page that runs in your browser and lets you enter hardware specs, like all of the other tools like this. It is not a good idea to install and run unknown projects like this on your computer in this age. 3. The project is very obviously vibecoded, down to the…

And the worst part is that it gets upvoted to the front page, while other hard work projects and articles don't. There's tons of vote scam sites where you can purchase mass upvotes on here as well as other platforms. There's no real repercussions if caught doing that :/

Re: Show HN: Find the best local LLM for your hardware, ranked by benchmarks

#75

Post by Jasssss »

andyyyy64 wrote: post #23
Jasssss wrote: post #2

The plan command is clever. How do you handle the VRAM estimation for models with sliding window attention vs full context? Something like Mistral at 32k context uses way less KV cache than Llama at the same context length, but from the README it looks like the estimation is based on a fixed context size. Does it account for that?

Good catch that's a real gap. The KV estimate is GQA/MQA-aware (per-model head config) but currently assumes dense full-context attention; it does not model sliding-window / chunked attention, so for SWA models like Mistral or Gemma at long context it over-estimates KV. The error is conservative — it tells you a model needs more than it does, not less, so it won't push you into an OOM — but it's still wrong. I'll ope…

Nice! Mistral 7B v0.1 is sliding_window: 4096 in the HuggingFace config.json (though v0.2 sets it to null). Gemma 2 alternates sliding window (4096) and full attention every other layer. Both have the field in the model config so maybe you could pull it from the same API you're already using.

Re: Show HN: Find the best local LLM for your hardware, ranked by benchmarks

#76

Post by jdiff »

kilroy123 wrote: post #17
jordiburgos wrote: post #10

This is very helpful too: https://www.canirun.ai/

Yes, I really like this site too, but it's a bit outdated. "39d ago" in AI time is like 1 year outdated info.

I don't think this has been the case for at least 39 days. The news is slowing down. The big headlines now, besides unverified marketing claims, are efficiency gains. Which are fantastic, but don't seem to be met with matching performance gains.

Re: Show HN: Find the best local LLM for your hardware, ranked by benchmarks

#77

Post by wren6991 »

dlcarrier wrote: post #74
wren6991 wrote: post #55

I also have a script to find the best LLM for your hardware. Here: echo "Qwen3.6-27B"

What about for 16 GB VRAM? Is Qwen3.5-9B worthwhile?

For 16 GB I would look into running Qwen3.6-35B-A3B (MoE) with some layers offloaded to CPU.

Re: Show HN: Find the best local LLM for your hardware, ranked by benchmarks

#79

Post by jdiff »

freeCandy wrote: post #16
jordiburgos wrote: post #10

This is very helpful too: https://www.canirun.ai/

Every browser gives me a different result, I guess I can't blame the site for that. But it should perhaps mention which browser would be the most accurate.

Memory bandwidth is completely different on any browser from measured results on my M2 Pro machine. Weirdly, the estimated performance levels and even exact product name differ between Chrome and Firefox. Firefox calls it an M2 Pro and overshoots measured memory bandwidth by 40GB/S, Chrome calls it an "Apple M2 Pro" and overshoots by 80.
Post reply on HN