GGML – AI at the Edge
ggml.ai
GGML – AI at the Edge
1–10 of 246 posts
Re: GGML – AI at the Edge
#2How common is avx on edge platforms?
Re: GGML – AI at the Edge
#3[deleted]
Re: GGML – AI at the Edge
#4This guy is damned good. I sponsored him on Github because his software is dope. I also like how when some controversy erupted on the project he just ejected the controversial people and moved on. Good stewardship. Great code.
I recall something like when he first ported it and it worked on my M1 Max he hadn't even yet tested it on Apple Silicon since he didn't have the hardware.
Honestly, with this and whisper, I am a huge fan. Good luck to him and the new company.
Re: GGML – AI at the Edge
#5I believe ggml is the basis of llama.cpp (the OP says it's "used by llama.cpp")? I don't know much about either, but when I read the llama.cpp code to see how it was created so quickly, I got the sense that the original project was ggml, given the amount of pasted code I saw. It seemed like quite an impressive library.
Re: GGML – AI at the Edge
#6> Nat Friedman and Daniel Gross provided the pre-seed funding.
Why? Why should VCs get involved again?
They are just going to look for an exit and end up getting acquired by Apple Inc.
Not again.
Re: GGML – AI at the Edge
#7I believe ggml is the basis of llama.cpp (the OP says it's "used by llama.cpp")? I don't know much about either, but when I read the llama.cpp code to see how it was created so quickly, I got the sense that the original project was ggml, given the amount of pasted code I saw. It seemed like quite an impressive library.
https://news.ycombinator.com/item?id=33877893
“OpenAI recently released a model for automatic speech recognition called Whisper. I decided to reimplement the inference of the model from scratch using C/C++. To achieve this I implemented a minimalistic tensor library in C and ported the high-level architecture of the model in C++.”
That “minimalistic tensor library” was ggml.
Re: GGML – AI at the Edge
#8> Nat Friedman and Daniel Gross provided the pre-seed funding. Why? Why should VCs get involved again? They are just going to look for an exit and end up getting acquired by Apple Inc. Not again.
It's possible to do whatever you want without VCs. The code is open source so you can start where he's starting from and run a purely different enterprise if you desire.
Re: GGML – AI at the Edge
#9Does GGML support training on the edge? We're especially interested in training support for Android+iOS
Re: GGML – AI at the Edge
#10> Nat Friedman and Daniel Gross provided the pre-seed funding. Why? Why should VCs get involved again? They are just going to look for an exit and end up getting acquired by Apple Inc. Not again.
Do you have pre-seed funding to give him?