Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GGML – AI at the Edge

ggml.ai

Post reply on HN
Print view View original

41–50 of 246 posts

Re: GGML – AI at the Edge

#41

Post by freedomben »

PrimeMcFly wrote: post #24
_20p0 wrote: post #4

This guy is damned good. I sponsored him on Github because his software is dope. I also like how when some controversy erupted on the project he just ejected the controversial people and moved on. Good stewardship. Great code. I recall something like when he first ported it and it worked on my M1 Max he hadn't even yet tested it on Apple Silicon since he didn't have the hardware. Honestly, with this and whisper, I am…

> I also like how when some controversy erupted on the project he just ejected the controversial people and moved on. Good stewardship Do you have more info on the controversy? I'm not sure ejecting developers just because of controversy is honestly good stewardship.

Right. More details needed to know if this is good stewardship (ejecting two toxic individuals) or laziness (ejecting a villain and a hero to get rid of the "problem" easily). TikTok was using this method for a while by ejecting both bullies and victims, and it "solved" the problem but most people see the injustice there.

I'm not saying it was bad stewardship, I honestly don't know. I just agree that we shouldn't make a judgment without more information.

Re: GGML – AI at the Edge

#42

Post by kgwgk »

evanwise wrote: post #18
_20p0 wrote: post #4

This guy is damned good. I sponsored him on Github because his software is dope. I also like how when some controversy erupted on the project he just ejected the controversial people and moved on. Good stewardship. Great code. I recall something like when he first ported it and it worked on my M1 Max he hadn't even yet tested it on Apple Silicon since he didn't have the hardware. Honestly, with this and whisper, I am…

What was the controversy?

https://news.ycombinator.com/item?id=35411909

Re: GGML – AI at the Edge

#44

Post by danielbln »

behnamoh wrote: post #39
samwillis wrote: post #19

ggml and llama.cpp are such a good platform for local LLMs, having some financial backing to support development is brilliant. We should be concentrating as much as possible to do local inference (and training) based on privet data. I want a local ChatGPT fine tuned on my personal data running on my own device, not in the cloud. Ideally open source too, llama.cpp is looking like the best bet to achieve that!

I wonder if ClosedAI and other companies use the findings of the open source community in their products. For example, do they use QLORA to reduce the costs of training and inference? Do they quantize their models to serve non-subscribing consumers?

Not disagreeing with your points, but saying "ClosedAI" is about as clever as writing M$ for Microsoft back in the day, which is to say not very.

Re: GGML – AI at the Edge

#45

Post by okhuman »

The establishment of ggml.ai a company focusing ggml and llama.cpp, the most innovative and exciting platform to come for local LLMs, on a Open Core model is just laziness.

Just because you can (and have the connections), doesn't mean you should. It's a sad state of OSS when the best most brightest developers/founders reach for antiquated models.

Maybe we take up a new rules in OSS communities that say you must release your CORE software as MIT at the same time you plan to go Open Core (and no sooner).

Why should OSS communities take on your product market fit?!

Re: GGML – AI at the Edge

#46

Post by ulchar »

rvz wrote: post #37
samwillis wrote: post #19

ggml and llama.cpp are such a good platform for local LLMs, having some financial backing to support development is brilliant. We should be concentrating as much as possible to do local inference (and training) based on privet data. I want a local ChatGPT fine tuned on my personal data running on my own device, not in the cloud. Ideally open source too, llama.cpp is looking like the best bet to achieve that!

> ggml and llama.cpp are such a good platform for local LLMs, having some financial backing to support development is brilliant The problem is, this financial backing and support is via VCs, who will steer the project to close it all up again. > I want a local ChatGPT fine tuned on my personal data running on my own device, not in the cloud. Ideally open source too, llama.cpp is looking like the best bet to achieve t…

> The problem is, this financial backing and support is via VCs, who will steer the project to close it all up again.

How exactly could they meaningfully do that? Genuine question. The issue with the OpenAI business model is that the collaboration within academia and open source circles is creating innovations that are on track to out-pace the closed source approach. Does OpenAI have the pockets to buy the open source collaborators and researchers?

I'm truly cynical about many aspects of the tech industry but this is one of those fights that open source could win for the betterment of everybody.

Re: GGML – AI at the Edge

#47

Post by wmf »

okhuman wrote: post #45

The establishment of ggml.ai a company focusing ggml and llama.cpp, the most innovative and exciting platform to come for local LLMs, on a Open Core model is just laziness. Just because you can (and have the connections), doesn't mean you should. It's a sad state of OSS when the best most brightest developers/founders reach for antiquated models. Maybe we take up a new rules in OSS communities that say you must relea…

This looks off-topic since GGML has not announced anything about open core and their software is already MIT.

More generally, if you want to take away somebody's business model you need to provide one that works. It isn't easy.

Re: GGML – AI at the Edge

#48

Post by wmf »

okhuman wrote: post #26
rvz wrote: post #6

> Nat Friedman and Daniel Gross provided the pre-seed funding. Why? Why should VCs get involved again? They are just going to look for an exit and end up getting acquired by Apple Inc. Not again.

+1. VC involvement in projects like these always pivot the team away from the core competency of what you'd expect them to deliver - into some commercialization aspect that convert only a tiny fraction of the community yet take up 60%+ of the core developer team's time. I don't know why project founders head this way...as the track records of leaders who do this end up disappointing the involved community at some poi…

Developers shouldn't be unpaid slaves to the community.

Re: GGML – AI at the Edge

#50

Post by loa_in_ »

danielbln wrote: post #44
behnamoh wrote: post #39

Earlier quoted context omitted.

I wonder if ClosedAI and other companies use the findings of the open source community in their products. For example, do they use QLORA to reduce the costs of training and inference? Do they quantize their models to serve non-subscribing consumers?

Not disagreeing with your points, but saying "ClosedAI" is about as clever as writing M$ for Microsoft back in the day, which is to say not very.

I'd say saying M$ makes it harder for M$ to find out I'm talking about them in them in the indexed web because it's more ambiguous, that's all I need to know.
Post reply on HN