Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

github.com

Post reply on HN
Print view View original

211–220 of 241 posts

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#211

Post by seanmcdirmid »

throwaway27448 wrote: post #154
razster wrote: post #108

Earlier quoted context omitted.

My fear is that these large "AI" companies will lobby to have these open source options removed or banned, growing concern. I'm not sure how else to explain how much I enjoy using what HF provides, I religiously browse their site for new and exciting models to try.

They can try. I don't think they'll be able to get the toothpaste back in the tube. The data will just move our of the country.

Many of the models on hugging face are already Chinese. It’s kind of obvious that local AI is going to flourish more in China than the USA due to hardware constraints.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#212

Post by seanmcdirmid »

BoredomIsFun wrote: post #191
culi wrote: post #152

Earlier quoted context omitted.

Mistral is cool and I wish them success but it consistently ranks extremely low on benchmarks while still being expensive. Chinese models like DeepSeek might rank almost as low as Mistral but they are significantly cheaper. And Kimi is the best of both worlds with incredible benchmark results while still being incredibly cheap I know things change rapidly so I'm not counting them out quite yet but I don't see them as…

> it consistently ranks extremely low on benchmarks As general purpose chatbots small Mistral models are better than comparably sized Chiniese models, as they have better SimpleQA scores and general knowledge of Western culture.

It’s really hard to beat qwen coder, especially for role play where the instruction following is really useful. I don’t think their corpus is lacking in western knowledge, although I wonder if Chinese users get even better results from it?

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#213

Post by seanmcdirmid »

culi wrote: post #188
Eupolemos wrote: post #163

Earlier quoted context omitted.

Why are you talking price when we are talking local AI? That doesn't make any sense to me. Am I missing something?

Your electricity is free?

Apple silicon is crazy efficient as well as being comparable to GPUs in performance for max and ultra chips.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#214

Post by BoredomIsFun »

seanmcdirmid wrote: post #212
BoredomIsFun wrote: post #191

Earlier quoted context omitted.

> it consistently ranks extremely low on benchmarks As general purpose chatbots small Mistral models are better than comparably sized Chiniese models, as they have better SimpleQA scores and general knowledge of Western culture.

It’s really hard to beat qwen coder, especially for role play where the instruction following is really useful. I don’t think their corpus is lacking in western knowledge, although I wonder if Chinese users get even better results from it?

> It’s really hard to beat qwen coder, for role play

I am not sure if you actually tried that. Mistrals are widely asccepted go-to models for roleplay and creative writing. No Qwens are good at prose, except for their latest big Qwen 3.5.

> I don’t think their corpus is lacking in western knowledge,

It absolutely does, especially pop culture knowledge.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#215

Post by Eisenstein »

imiric wrote: post #116
llm_nerd wrote: post #109

Earlier quoted context omitted.

HN goes through phases. I remember when patio11 was the star of the hour on here. At another time it was that security guy (can't remember his name). And for those who think it's just organic with all of the upvotes, HN absolutely does have a +/- comment bias for users, and it does automatically feature certain people and suppress others.

> And for those who think it's just organic with all of the upvotes, HN absolutely does have a bias for authors, and it does automatically feature certain people and suppress others. Exactly. There are configurable settings for each account, which might be automatically or manually set—I'm not sure–, that control the initial position of a comment in threads, and how long it stays there. There might be a reward system…

There is a saying that if everyone you encounter seems to be unreasonable, maybe it isn't the other people that are being unreasonable.

This isn't to say that social media is fair, or that people vote properly or that any ranking system based on agreement by readers is a good one. However, generally when you are getting negativity communicated to you and you are seeing consistently poor results around actions you take, it is going to be useful to examine the possibility that there is a difference in how you perceive what you are doing vs how others do. In that case spending time trying to figure out ways in which you are being wronged so that you can continue in the same manner is going to be time wasted.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#216

Post by BoredomIsFun »

0xbadcafebee wrote: post #88
tkp-415 wrote: post #25

Can anyone point me in the direction of getting a model to run locally and efficiently inside something like a Docker container on a system with not so strong computing power (aka a Macbook M1 with 8gb of memory)? Is my only option to invest in a system with more computing power? These local models look great, especially something like https://huggingface.co/AlicanKiraz0/Cybersecurity-BaronLLM_O... for assisting in p…

8GB is not enough to do complex reasoning, but you could do very small simple things. Models like Whisper, SmolVLM, Quen2.5-0.5B, Phi-3-mini, Granite-4.0-micro, Mistral-7B, Gemma3, Llama-3.2 all work on very little memory. Tiny models can do a lot if you tune/train them. They also need to be used differently: system prompt preloaded with information, few-shot examples, reasoning guidance, single-task purpose, strict…

> Mistral-7B

Is hopelessly dated. There are much better newer models around.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#217

Post by seanmcdirmid »

BoredomIsFun wrote: post #214
seanmcdirmid wrote: post #212

Earlier quoted context omitted.

It’s really hard to beat qwen coder, especially for role play where the instruction following is really useful. I don’t think their corpus is lacking in western knowledge, although I wonder if Chinese users get even better results from it?

> It’s really hard to beat qwen coder, for role play I am not sure if you actually tried that. Mistrals are widely asccepted go-to models for roleplay and creative writing. No Qwens are good at prose, except for their latest big Qwen 3.5. > I don’t think their corpus is lacking in western knowledge, It absolutely does, especially pop culture knowledge.

Instruct and coder just follow instructions so well though. I guess I’ve just never been able to make mistral work well, I guess.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#218

Post by imiric »

Eisenstein wrote: post #215
imiric wrote: post #116

Earlier quoted context omitted.

> And for those who think it's just organic with all of the upvotes, HN absolutely does have a bias for authors, and it does automatically feature certain people and suppress others. Exactly. There are configurable settings for each account, which might be automatically or manually set—I'm not sure–, that control the initial position of a comment in threads, and how long it stays there. There might be a reward system…

There is a saying that if everyone you encounter seems to be unreasonable, maybe it isn't the other people that are being unreasonable. This isn't to say that social media is fair, or that people vote properly or that any ranking system based on agreement by readers is a good one. However, generally when you are getting negativity communicated to you and you are seeing consistently poor results around actions you tak…

How are you getting persecution complex from what I said? If anything, your comment might be feeding that delusion. :)

My point is that HN definitely has certain weights associated with accounts, which control the karma, visibility, and ultimately discussion of certain topics.

This problem doesn't affect only negativity or downvotes, but upvotes as well. The most upvoted comments are not necessarily of the highest quality, or contribute the most to the discussion. They just happen to be the most visible, and to generally align with the feeling of the hive mind.

I know this because some of my own comments have been at the top, without being anything special, while others I think are, barely get any attention. I certainly examine my thinking whenever it strongly aligns with the hive mind, as this community does not particularly align with my values.

I also tend to seek out comments near the bottom of threads, and have dead comments enabled, precisely to counteract this flawed system. I often find quality opinions there, so I suggest everyone do the same as well.

An essential feature of a healthy and interesting discussion forum is to accomodate different viewpoints. That starts by not burying those that disagree with the majority, or boosting those that agree. AFAIK no online system has gotten this right yet.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#219

Post by BoredomIsFun »

seanmcdirmid wrote: post #217
BoredomIsFun wrote: post #214

Earlier quoted context omitted.

> It’s really hard to beat qwen coder, for role play I am not sure if you actually tried that. Mistrals are widely asccepted go-to models for roleplay and creative writing. No Qwens are good at prose, except for their latest big Qwen 3.5. > I don’t think their corpus is lacking in western knowledge, It absolutely does, especially pop culture knowledge.

Instruct and coder just follow instructions so well though. I guess I’ve just never been able to make mistral work well, I guess.

Qwen3 30B A3B and that big 400+ B Coder were absolutely terrible at editing fiction. I would tell them what to change in the prose and they'd just regurgitate text with no changes.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#220

Post by Zetaphor »

fc417fc802 wrote: post #187
Zetaphor wrote: post #184

Earlier quoted context omitted.

ATT once told me if I don't pay for their TV service then my home gigabit fiber would have a 1TB cap. They had an agreement with the apartment building so I had no other choice of provider.

Buy our off brand netflix or else we'll make it so you can't watch netflix. How is that legal?

The law is written by the highest bidder, and the telecom lobbyists are very generous
Post reply on HN