Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

github.com

Post reply on HN
Print view View original

51–60 of 241 posts

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#51

Post by embedding-shape »

disiplus wrote: post #36
option wrote: post #29

Isn't HF banned in China? Also, how are many Chinese labs on Twitter all the time? In either case - huge thanks to them for keeping AI open!

I think in the West we think everything is blocked. But for example, if you book an eSIM, when you visit you already get direct access to Western services because they route it to some other server. Hong Kong is totally different: they basically use WhatsApp and Google Maps, and everything worked when I was there.

But also yes, parent is right, HF is more or less inaccessible, and Modelscope frequently cited as the mirror to use (although many Chinese labs seems to treat HF as the mirror, and Modelscope as the "real" origin).

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#52

Post by heliumtera »

Fin_Code wrote: post #48
mythz wrote: post #18

I consider HuggingFace more "Open AI" than OpenAI - one of the few quiet heroes (along with Chinese OSS) helping bring on-premise AI to the masses. I'm old enough to remember when traffic was expensive, so I've no idea how they've managed to offer free hosting for so many models. Hopefully it's backed by a sustainable business model, as the ecosystem would be meaningfully worse without them. We still need good value…

I still don't know why they are not running on torrent. Its the perfect use case.

How can you be the man in the middle in a truly P2P environment?

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#53

Post by freedomben »

Fin_Code wrote: post #48
mythz wrote: post #18

I consider HuggingFace more "Open AI" than OpenAI - one of the few quiet heroes (along with Chinese OSS) helping bring on-premise AI to the masses. I'm old enough to remember when traffic was expensive, so I've no idea how they've managed to offer free hosting for so many models. Hopefully it's backed by a sustainable business model, as the ecosystem would be meaningfully worse without them. We still need good value…

I still don't know why they are not running on torrent. Its the perfect use case.

That would shut out most people working for big corp, which is probably a huge percentage of the user base. It's dumb, but that's just the way corp IT is (no torrenting allowed).

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#54

Post by zozbot234 »

freedomben wrote: post #53
Fin_Code wrote: post #48

Earlier quoted context omitted.

I still don't know why they are not running on torrent. Its the perfect use case.

That would shut out most people working for big corp, which is probably a huge percentage of the user base. It's dumb, but that's just the way corp IT is (no torrenting allowed).

It's a sensible option, even when not everyone can really use it. Linux distros are routinely transfered via torrent, so why not other massive, open-licensed data?

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#55

Post by echoangle »

logicallee wrote: post #30
dmezzetti wrote: post #14

This is really great news. I've been one of the strongest supporters of local AI dedicating thousands of hours towards building a framework to enable it. I'm looking forward to seeing what comes of it!

>I've been one of the strongest supporters of local AI, dedicating thousands of hours towards building a framework to enable it. Sounds like you're very serious about supporting local AI. I have a query for you (and anyone else who feels like donating) about whether you'd be willing to donate some memory/bandwidth resources p2p to hosting an offline model: We have a local model we would like to distribute but don't h…

Maybe stupid question but why not just put it in a torrent?

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#56

Post by heliumtera »

HanClinto wrote: post #7

I'm regularly amazed that HuggingFace is able to make money. It does so much good for the world. How solid is its business model? Is it long-term viable? Will they ever "sell out"?

>Will they ever "sell out"?

Oh no, never. Don't worry, the usual investors are very well known for fighting for user autonomy (AMD, Nvidia, Intel,IBM, Qualcomm)

They are all very pro consumers and all backers are certainly here for your enjoyment only

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#57

Post by julien_c »

dmezzetti wrote: post #15
HanClinto wrote: post #7

I'm regularly amazed that HuggingFace is able to make money. It does so much good for the world. How solid is its business model? Is it long-term viable? Will they ever "sell out"?

They have paid hosting - https://huggingface.co/enterprise and paid accounts. Also consulting services. Seems like a pretty good foundation to me.

and a lot of traction on paid (private in particular) storage these days; sneak peek at new landing page: https://huggingface.co/storage

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#58

Post by sowbug »

embedding-shape wrote: post #47
sowbug wrote: post #39

Earlier quoted context omitted.

Why doesn't HF support BitTorrent? I know about hf-torrent and hf_transfer, but those aren't nearly as accessible as a link in the web UI.

> Why doesn't HF support BitTorrent? Harder to track downloads then. Only when clients hit the tracker would they be able to get download states, and forget about private repositories or the "gated" ones that Meta/Facebook does for their "open" models. Still, if vanity metrics wasn't so important, it'd be a great option. I've even thought of creating my own torrent mirror of HF to provide as a public service, as even…

I thought of the tracking and gate questions, too, when I vibed up an HF torrent service a few nights ago. (Super annoying BTW to have to download the files just to hash the parts, especially when webseeds exist.) Model owners could disable or gate torrents the same way they gate the models, and HF could still measure traffic by .torrent downloads and magnet clicks.

It's a bit like any legalization question -- the black market exists anyway, so a regulatory framework could bring at least some of it into the sunlight.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#59

Post by zozbot234 »

heliumtera wrote: post #56
HanClinto wrote: post #7

I'm regularly amazed that HuggingFace is able to make money. It does so much good for the world. How solid is its business model? Is it long-term viable? Will they ever "sell out"?

>Will they ever "sell out"? Oh no, never. Don't worry, the usual investors are very well known for fighting for user autonomy (AMD, Nvidia, Intel,IBM, Qualcomm) They are all very pro consumers and all backers are certainly here for your enjoyment only

These are all big hardware firms, which makes a lot of sense as a classic 'commoditize the complement' play. Not exactly pro-consumer, but not quite anti-consumer either!

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#60

Post by HanClinto »

tkp-415 wrote: post #25

Can anyone point me in the direction of getting a model to run locally and efficiently inside something like a Docker container on a system with not so strong computing power (aka a Macbook M1 with 8gb of memory)? Is my only option to invest in a system with more computing power? These local models look great, especially something like https://huggingface.co/AlicanKiraz0/Cybersecurity-BaronLLM_O... for assisting in p…

Maybe check out Docker Model Runner -- it's built on llama.cpp (in a good way -- not like Ollama) and handles I think most of what you're looking for?

https://www.docker.com/blog/run-llms-locally/

As far as how to find good models to run locally, I found this site recently, and I liked the data it provides:

https://localclaw.io/

Post reply on HN