What's in a GGUF, besides the weights – and what's still missing?
1–10 of 65 posts
Re: What's in a GGUF, besides the weights – and what's still missing?
#2Nice, I recently pulled down TheBloke 7B mistral to try out I have a 4070.
Re: What's in a GGUF, besides the weights – and what's still missing?
#3Nice, I recently pulled down TheBloke 7B mistral to try out I have a 4070.
I have a 2070 and can confirm it works amazingly fast.
I love TheBloke I wish he still made stuff
Re: What's in a GGUF, besides the weights – and what's still missing?
#4Nice, I recently pulled down TheBloke 7B mistral to try out I have a 4070.
I have a 2070 and can confirm it works amazingly fast. I love TheBloke I wish he still made stuff
What do you use it for? I'm still trying to use agents, I barely use copilot, only at work when I have to.
I didn't want to get personal with an LLM unless it was local so that's why I was setting this up but yeah. So far just research is what I was looking at.
Re: What's in a GGUF, besides the weights – and what's still missing?
#5>Published May 18, 2026
hmmm...
Re: What's in a GGUF, besides the weights – and what's still missing?
#6>Published May 18, 2026 hmmm...
whoops, my bad. Just a typo in the markdown. Fixed :)
Re: What's in a GGUF, besides the weights – and what's still missing?
#7Nice, I recently pulled down TheBloke 7B mistral to try out I have a 4070.
I love mistral, but that model is... not the best. Maybe try out Gemma 4 e4b, it's a similar size to Mistral 7B, and should run great on your 4070 ("E4B" is slightly misleading naming).
Re: What's in a GGUF, besides the weights – and what's still missing?
#8Nice, I recently pulled down TheBloke 7B mistral to try out I have a 4070.
I have a 2070 and can confirm it works amazingly fast. I love TheBloke I wish he still made stuff
Yeah, TheBloke era of local LLMs were good times. TBF Unsloth are doing a fantastic job of publishing quants of the major models quickly - they just don't have nearly the volume of "weird" models as TheBloke did.
Re: What's in a GGUF, besides the weights – and what's still missing?
#9Nice, I recently pulled down TheBloke 7B mistral to try out I have a 4070.
I love mistral, but that model is... not the best. Maybe try out Gemma 4 e4b, it's a similar size to Mistral 7B, and should run great on your 4070 ("E4B" is slightly misleading naming).
Thanks for the tip, what do you use Gemma 4 e4b for?
Re: What's in a GGUF, besides the weights – and what's still missing?
#10Earlier quoted context omitted.
I love mistral, but that model is... not the best. Maybe try out Gemma 4 e4b, it's a similar size to Mistral 7B, and should run great on your 4070 ("E4B" is slightly misleading naming).
Thanks for the tip, what do you use Gemma 4 e4b for?
some say it’s a miniaturized gemini model
it’s good at writing, coding, decently intelligent
you can try it on nvidia nim