Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

github.com

Post reply on HN
Print view View original

61–70 of 241 posts

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#61

Post by HanClinto »

logicallee wrote: post #30
dmezzetti wrote: post #14

This is really great news. I've been one of the strongest supporters of local AI dedicating thousands of hours towards building a framework to enable it. I'm looking forward to seeing what comes of it!

>I've been one of the strongest supporters of local AI, dedicating thousands of hours towards building a framework to enable it. Sounds like you're very serious about supporting local AI. I have a query for you (and anyone else who feels like donating) about whether you'd be willing to donate some memory/bandwidth resources p2p to hosting an offline model: We have a local model we would like to distribute but don't h…

Hosting model weights for projects like this I think is something that you could upload to a space in Hugging Face?

What services would you need that Hugging Face doesn't provide?

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#62

Post by freedomben »

zozbot234 wrote: post #54
freedomben wrote: post #53

Earlier quoted context omitted.

That would shut out most people working for big corp, which is probably a huge percentage of the user base. It's dumb, but that's just the way corp IT is (no torrenting allowed).

It's a sensible option, even when not everyone can really use it. Linux distros are routinely transfered via torrent, so why not other massive, open-licensed data?

Oh as an option, yeah I agree it makes a ton of sense. I just would expect a very, very small percentage of people to use the torrent over the direct download. With Linux distros, the vast majority of downloads still come from standard web servers. When I download distro images I opt for torrents, but very few people do the same

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#63

Post by embedding-shape »

sowbug wrote: post #58
embedding-shape wrote: post #47

Earlier quoted context omitted.

> Why doesn't HF support BitTorrent? Harder to track downloads then. Only when clients hit the tracker would they be able to get download states, and forget about private repositories or the "gated" ones that Meta/Facebook does for their "open" models. Still, if vanity metrics wasn't so important, it'd be a great option. I've even thought of creating my own torrent mirror of HF to provide as a public service, as even…

I thought of the tracking and gate questions, too, when I vibed up an HF torrent service a few nights ago. (Super annoying BTW to have to download the files just to hash the parts, especially when webseeds exist.) Model owners could disable or gate torrents the same way they gate the models, and HF could still measure traffic by .torrent downloads and magnet clicks. It's a bit like any legalization question -- the bl…

> Model owners could disable or gate torrents the same way they gate the models, and HF could still measure traffic by .torrent downloads and magnet clicks.

But that'll only stop a small part, anyone could share the infohash and if you're using the dht/magnet without .torrent files or clicks on a website, no one can count those downloads unless they too scrape the dht for peers who are reporting they've completed the download.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#65

Post by jackbravo »

microsoftedging wrote: post #46
HanClinto wrote: post #7

I'm regularly amazed that HuggingFace is able to make money. It does so much good for the world. How solid is its business model? Is it long-term viable? Will they ever "sell out"?

FT had a solid piece a few weeks back: "Why AI start-up Hugging Face turned down a $500mn Nvidia deal" https://giftarticle.ft.com/giftarticle/actions/redeem/9b4eca...

sounds very interesting, but even though it says giftarticle.ft, I got blocked by a paywall.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#66

Post by sowbug »

embedding-shape wrote: post #63
sowbug wrote: post #58

Earlier quoted context omitted.

I thought of the tracking and gate questions, too, when I vibed up an HF torrent service a few nights ago. (Super annoying BTW to have to download the files just to hash the parts, especially when webseeds exist.) Model owners could disable or gate torrents the same way they gate the models, and HF could still measure traffic by .torrent downloads and magnet clicks. It's a bit like any legalization question -- the bl…

> Model owners could disable or gate torrents the same way they gate the models, and HF could still measure traffic by .torrent downloads and magnet clicks. But that'll only stop a small part, anyone could share the infohash and if you're using the dht/magnet without .torrent files or clicks on a website, no one can count those downloads unless they too scrape the dht for peers who are reporting they've completed the…

Right, but that's already happening today. That's the black-market point.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#67

Post by csunoser »

the__alchemist wrote: post #19

Does anyone have a good comparison of HuggingFace/Candle to Burn? I am testing them concurrently, and Burn seems to have an easier-to-use API. (And can use Candle as a backend, which is confusing) When I ask on Reddit or Discord channels, people overwhelmingly recommend Burn, but provide no concrete reasons beyond "Candle is more for inference while Burn is training and inference". This doesn't track, as I've done tr…

I have used both (albeit 2 years ago, and things change really fast). At the time, Candle didn't have 2d conv backprop with strides properly implemented. And getting Burn running libtch backend was just a lot simpler.

I did use candle for wasm based inference for teaching purposes - that was reasonably painless and pretty nice.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#68

Post by nerevarthelame »

jackbravo wrote: post #65
microsoftedging wrote: post #46

Earlier quoted context omitted.

FT had a solid piece a few weeks back: "Why AI start-up Hugging Face turned down a $500mn Nvidia deal" https://giftarticle.ft.com/giftarticle/actions/redeem/9b4eca...

sounds very interesting, but even though it says giftarticle.ft, I got blocked by a paywall.

https://archive.is/zSyUc

To summarize, they rejected Nvidia's offer because they didn't want one outsized investor who could sway decisions. And "the company was also able to turn down Nvidia due to its stable finances. Hugging Face operates a 'freemium' business model. Three per cent of customers, usually large corporations, pay for additional features such as more storage space and the ability to set up private repositories."

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#69

Post by logicallee »

echoangle wrote: post #55
logicallee wrote: post #30

Earlier quoted context omitted.

>I've been one of the strongest supporters of local AI, dedicating thousands of hours towards building a framework to enable it. Sounds like you're very serious about supporting local AI. I have a query for you (and anyone else who feels like donating) about whether you'd be willing to donate some memory/bandwidth resources p2p to hosting an offline model: We have a local model we would like to distribute but don't h…

Maybe stupid question but why not just put it in a torrent?

Torrents require users to download and install a torrent client! In addition, we would like to retain the possibility of giving live updates to the latest version of a sovereign fine-tuned file, torrents don't autoupdate. We want to keep improving what people get.

Finally, we would like the possibility of setting up market dynamics in the future: if you aren't currently using all your ram, why not rent it out? This matches the p2p edge architecture we envision.

In addition, our work on WebGPU would allow you to rent out your gpu to a background tab whenever you're not using it. Why have all that silicon sit idle when you could rent it out?

You could also donate it to help fine tune our own sovereign model.

All of this will let us bootstrap to the point where we could be trusted with a download.

We have a rather paranoid approach to security.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#70

Post by logicallee »

liuliu wrote: post #49
logicallee wrote: post #30

Earlier quoted context omitted.

>I've been one of the strongest supporters of local AI, dedicating thousands of hours towards building a framework to enable it. Sounds like you're very serious about supporting local AI. I have a query for you (and anyone else who feels like donating) about whether you'd be willing to donate some memory/bandwidth resources p2p to hosting an offline model: We have a local model we would like to distribute but don't h…

> We have a local model we would like to distribute but don't have a good CDN. That is not true. I am serving models off Cloudflare R2. It is 1 petabyte per month in egress use and I basically pay peanuts (~$200 everything included).

1 petabyte per month is 1 million downloads of a 1 GB file. We intend to scale to more than 1 million downloads per month. We have a specific scaling architecture in mind. We're qualified to say this because we've ported a billion parameter model to run in your browser - fast - on either webgpu or wasm. (You can see us doing it live at the youtube link in my comment above.) There is a lot of demand for that.
Post reply on HN