There's so much potential for this as a tech powering all sorts of products, I hope it doesn't just become some Tarsnap type thing. (judging from initial site)
What's wrong with "Tarsnap type things"?
GGML – AI at the Edge
201–210 of 246 posts
Re: GGML – AI at the Edge
#202Running whisper locally on my iPhone back in December and watching perfect transcriptions pop out without sending anything to a server was a real lightbulb moment for me that set in motion a bunch of the work I’m doing now. Excited to see the new heights this unlocks!
What are you using to run Whisper locally?
Re: GGML – AI at the Edge
#203Georgi if you're reading this, I've had a lot of fun with whisper.cpp llama.cpp because of you so thank you very much.
I envy his drive and ambition. I can't force myself to finish writing a simple alarm clock app for android, never mind pathing the literal road to the future of Open Source AI.
Would someone else have taken his place had he not been around? Maybe, but I'm insanely happy that he is around.
The amount of hours I've sunk into LLM's is crazy, and it's mostly thanks to his work that I can both run and download models in meaningful timeframes.
And yes, I have tested llama.cpp on my android and it works 100% on termux. (Your biggest enemy here will be Android process reaper when you hit the memory cap)
Re: GGML – AI at the Edge
#204Re: GGML – AI at the Edge
#205Earlier quoted context omitted.
[flagged]
Are you talking about Slaren? I wrote a blog post promoting him a while back. You can read it here: https://justine.lol/mmap/ It also spotlights other unsung heroes who played important roles in helping me bring mmap() to the machine learning community. As for me being trans, the problem is that 4chan does care. The /g/ local model discussion board was consumed with discussing my gender status for more than a month.…
>I wrote a blog post promoting him a while back.
That's about 2 weeks after the drama around PR 613, which you factually touted as "your work" in several different places.
Re: GGML – AI at the Edge
#206> Nat Friedman and Daniel Gross provided the pre-seed funding. Why? Why should VCs get involved again? They are just going to look for an exit and end up getting acquired by Apple Inc. Not again.
Daniel Gross is a good guy, a yes his company did get acquired by apple a while back, but he loves to foster really dope stuff by amazing people, and ggml certainly fits the bill. And this looks like an Angel investment, not a VC one if that makes any difference to you.
"Good" is subjective, I guess.
Daniel Gross set up a company that seemed akin to indebted servitude, modern day slavery. They called it "Pioneer" and later changed the terms, I guess because of backlash.
They gave you a little bit of money to "do whatever you want" but owned a huge stake in anything you did in the future for a long period of time. They didn't advertise that part very heavily, they mostly portrayed it as "we're doing this because we're philanthropists" imo, rather than because they wanted to reinvent indentured servitude within the modern legal framework.
Why do I write these posts? Because I desperately want to believe we can get rich without doing dishonest, evil things. Maybe I'm wrong. Maybe that's why all these guys behave this way. Maybe it really is never enough.
Re: GGML – AI at the Edge
#207Earlier quoted context omitted.
It makes syncing between llama.cpp, whisper.cpp, and ggml itself quite straightforward. I think the lesson here is that this setup has enabled some very high-speed project evolution or, at least, not got in its way. If that is surprising and you were expecting downsides, a) why; and b) where did they go?
https://git-scm.com/book/en/v2/Git-Tools-Submodules
Or... `cp`. It's fine.
Re: GGML – AI at the Edge
#208I've always thought on the edge to be IoT type stuff. So running on embedded devices. But maybe that not the case?
Here it means more 'on your own device' rather than 'in the cloud'
You could consider that the real edge, whereas edge computing often means 'at the edge of the cloud' i.e. local CDN node
Re: GGML – AI at the Edge
#209Does GGML support training on the edge? We're especially interested in training support for Android+iOS
What kind of applications do you see for training on mobile devices? Is anyone using this in industry?
Re: GGML – AI at the Edge
#210Can anyone explain to me, in simple terms, and at a high level, what the heck am I looking at? What is this library for? What does it mean "it is used by lama.cpp and whisper.cpp"? How is it revolutionary? Thank you very much in advance!
Thank you so much for your kindness Orost.
Sharing really IS caring.
I understand.
May good things happen to you. Peace.