Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

github.com

Post reply on HN
Print view View original

71–80 of 241 posts

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#71

Post by bee_rider »

nerevarthelame wrote: post #68
jackbravo wrote: post #65

Earlier quoted context omitted.

sounds very interesting, but even though it says giftarticle.ft, I got blocked by a paywall.

https://archive.is/zSyUc To summarize, they rejected Nvidia's offer because they didn't want one outsized investor who could sway decisions. And "the company was also able to turn down Nvidia due to its stable finances. Hugging Face operates a 'freemium' business model. Three per cent of customers, usually large corporations, pay for additional features such as more storage space and the ability to set up private rep…

Freemium seems to be working pretty well for them—what’s the alternative website, after all. They seem to command their niche.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#72

Post by zrm »

freedomben wrote: post #62
zozbot234 wrote: post #54

Earlier quoted context omitted.

It's a sensible option, even when not everyone can really use it. Linux distros are routinely transfered via torrent, so why not other massive, open-licensed data?

Oh as an option, yeah I agree it makes a ton of sense. I just would expect a very, very small percentage of people to use the torrent over the direct download. With Linux distros, the vast majority of downloads still come from standard web servers. When I download distro images I opt for torrents, but very few people do the same

With Linux distros they typically put the web link right on the main page and have a torrent available if you go look for it, because they want you to try their distro more than they want to save some bandwidth.

Suppose HF did the opposite because the bandwidth saved is more and they're not as concerned you might download a different model from someone else.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#73

Post by ukblewis »

Honestly I’m shocked to be the only one I see of this opinion: HuggingFace’s `accelerate`, `transformers` and `datasets` have been some of the worst open source Python libraries I have ever used that I had to use. They break backwards compatibility constantly, even on APIs which are not underscore/dunder named even on minor version releases without even documenting this, they refuse PRs fixing their lack of `overloads` type annotations which breaks type checking on their libraries and they just generally seem to have spaghetti code. I am not excited that another team is joining them and consolidating more engineering might in the hands of these people

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#74

Post by ukblewis »

ukblewis wrote: post #73

Honestly I’m shocked to be the only one I see of this opinion: HuggingFace’s `accelerate`, `transformers` and `datasets` have been some of the worst open source Python libraries I have ever used that I had to use. They break backwards compatibility constantly, even on APIs which are not underscore/dunder named even on minor version releases without even documenting this, they refuse PRs fixing their lack of `overload…

And I said all of that despite us continuing to use their platform and libraries extensively… We just don’t have a choice due to their dominance of open source ML

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#75

Post by ukblewis »

ukblewis wrote: post #73

Honestly I’m shocked to be the only one I see of this opinion: HuggingFace’s `accelerate`, `transformers` and `datasets` have been some of the worst open source Python libraries I have ever used that I had to use. They break backwards compatibility constantly, even on APIs which are not underscore/dunder named even on minor version releases without even documenting this, they refuse PRs fixing their lack of `overload…

And clearly I say all of this in my name and not my employers name

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#76

Post by 0xbadcafebee »

> The community will continue to operate fully autonomously and make technical and architectural decisions as usual. Hugging Face is providing the project with long-term sustainable resources, improving the chances of the project to grow and thrive. The project will continue to be 100% open-source and community driven as it is now.

I want this to be true, but business interests win out in the end. Llama.cpp is now the de-facto standard for local inference; more and more projects depend on it. If a company controls it, that means that company controls the local LLM ecosystem. And yeah, Hugging Face seems nice now... so did Google originally. If we all don't want to be locked in, we either need a llama.cpp competitor (with a universal abstration), or it should be controlled by an independent nonprofit.

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#79

Post by ontouchstart »

tkp-415 wrote: post #25

Can anyone point me in the direction of getting a model to run locally and efficiently inside something like a Docker container on a system with not so strong computing power (aka a Macbook M1 with 8gb of memory)? Is my only option to invest in a system with more computing power? These local models look great, especially something like https://huggingface.co/AlicanKiraz0/Cybersecurity-BaronLLM_O... for assisting in p…

This is the easiest set up on a Mac. You need at least 16gb on a MacBook:

https://github.com/ggml-org/llama.cpp/discussions/15396

Re: Ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

#80

Post by liuliu »

echoangle wrote: post #55
logicallee wrote: post #30

Earlier quoted context omitted.

>I've been one of the strongest supporters of local AI, dedicating thousands of hours towards building a framework to enable it. Sounds like you're very serious about supporting local AI. I have a query for you (and anyone else who feels like donating) about whether you'd be willing to donate some memory/bandwidth resources p2p to hosting an offline model: We have a local model we would like to distribute but don't h…

Maybe stupid question but why not just put it in a torrent?

It is very simple. Storage / bandwidth is not expensive. Residential bandwidth is. If you can convince people to install a bandwidth-related software on their residential homes, you can then charge other people $5 to $10 per 1GiB bandwidth (useful for botnet mostly, get around DDOS protections and other reCAPTCHA tasks).
Post reply on HN