What’s the best model right now that outperforms Qwopus3.6-27B-v2-MTP-GGUF 8-bit on a 24 GB VRAM GPU? Looking for real reviews. I found 4 bit not usable in production.
Ask HN: What's the best LLM model that on a 24 GB VRAM GPU?
1–4 of 4 posts
Re: Ask HN: What's the best LLM model that on a 24 GB VRAM GPU?
#2I would love to see Laguna.xs-2 on that, because it's very good on cpu only.
Re: Ask HN: What's the best LLM model that on a 24 GB VRAM GPU?
#3Good question. I was told in 2024 to get an RTX 3090 (24 GB VRAM), so I did, and nothing on HuggingFace was usable.
Re: Ask HN: What's the best LLM model that on a 24 GB VRAM GPU?
#4qwen3.6 27b and 35b work for me. I run them on proxmox with a an old P40 gpu, and ollama