I am happy for ggml team. They did so much work for quantization and actually made it available to everyone. Thank you.
Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI
161–170 of 241 posts
Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI
#162Earlier quoted context omitted.
[flagged]
> By law they are required to do whatever they can to maximize profits. I know it's a nit-pick, but I hate that this always gets brought up when it's not actually true. Public corporations face pressure from investors to maximize returns, sure, but there is no law stating that they have to maximize profits at all costs. Public companies can (and often do) act against the interest of immediate profits for some other g…
[flagged]
Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI
#163Earlier quoted context omitted.
Not true! Mistral is really really good, but I agree that there isn't a single decent open model from the USA.
Mistral is cool and I wish them success but it consistently ranks extremely low on benchmarks while still being expensive. Chinese models like DeepSeek might rank almost as low as Mistral but they are significantly cheaper. And Kimi is the best of both worlds with incredible benchmark results while still being incredibly cheap I know things change rapidly so I'm not counting them out quite yet but I don't see them as…
Why are you talking price when we are talking local AI?
That doesn't make any sense to me. Am I missing something?
Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI
#164[dead]
Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI
#165I don’t know if this warrants a separate thread here but I have to ask… How can I realistically get involved the AI development space? I feel left out with what’s going on and living in a bubble where AI is forced into by my employer to make use of it (GitHub Copilot), what is a realistic road map to kinda slowly get into AI development, whatever that means My background is full stack development in Java and React, a…
The competition for root and branch AI models and infrastructure is intense and skilled.
But if you're adjacent to some leaf use-case for AI, you're likely already as good as anyone else at productizing it.
And that's who is getting hired: people who show they can deliver product-market fit.
Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI
#166Earlier quoted context omitted.
i don't doubt this. i just find it questionable that one particular poster always gets in the spotlight when AI is the topic - while other conversations in my opinion offer more interesting angles.
Upvote the conversations that you find to be more interesting. If enough people do the same, they too will make it to the top.
Parent implies there might be some "boosting" involved, in which case, "upvote the conversations that you find to be more interesting" wont change anything...
Not saying this is the case, but it's what the comment implies, so "just upvote your faves" doesn't really address it.
Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI
#167Earlier quoted context omitted.
It's insane how much traffic HF must be pushing out of the door. I routinely download models that are hundreds of gigabytes in size from them. A fantastic service to the sovererign AI community.
My fear is that these large "AI" companies will lobby to have these open source options removed or banned, growing concern. I'm not sure how else to explain how much I enjoy using what HF provides, I religiously browse their site for new and exciting models to try.
How do you choose which models to try for which workflows? Do you have objective tests that you run, or do you just get a feel for them while using them in your daily workflow?
Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI
#168Earlier quoted context omitted.
At a certain point the energy starts to cost more than renting some GPUs.
Yeah, that is hard to argue with because I just go to OpenRouter and play around with a lot of models before I decide which ones I like. But there's something special about running it locally in your basement
I'd love to hear more about this. How do you decide that you like a model? For which use cases?
Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI
#169Earlier quoted context omitted.
Mistral have small variants (3B, 8B, 14B, etc.), as do others like IBM Granite and Qwen. Then there are finetunes based on these models, depending on your workflow/requirements.
True, but anything remotely useful is 300B and above
That is a very broad and silly position to take, especially in this thread.
I use Devstral 2 and Gemini 3 daily.
Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI
#170I have played with both mlx-lm and llama.cpp after I bought a 24GB M5 MacBook Pro last year.
Then I fell down the rabbit holes of uv, rust and C++ and forgot about LLMs. Today after I saw this announcement and answered someone’s question about how to set it up, when I got home, I decided play with llama.cpp again.
I was surprised and impressed:
https://ontouchstart.github.io/rabbit-holes/llama.cpp/
I am not going to use mlx-lm or lmstudio anymore. llama.cpp is so much fun.