There was a big stink one time when the file format changed, causing older model files to become unusable on newer versions of llama.cpp.
GGML – AI at the Edge
91–100 of 246 posts
Re: GGML – AI at the Edge
#92Earlier quoted context omitted.
Not disagreeing with your points, but saying "ClosedAI" is about as clever as writing M$ for Microsoft back in the day, which is to say not very.
I'd say saying M$ makes it harder for M$ to find out I'm talking about them in them in the indexed web because it's more ambiguous, that's all I need to know.
If we are talking about indexing, writing M$ is easier to find in an index because it is a such unique token. MS can mean many things (e.g. Miss), M$ is less ambiguous.
Re: GGML – AI at the Edge
#93Earlier quoted context omitted.
> I also like how when some controversy erupted on the project he just ejected the controversial people and moved on. Good stewardship Do you have more info on the controversy? I'm not sure ejecting developers just because of controversy is honestly good stewardship.
Right. More details needed to know if this is good stewardship (ejecting two toxic individuals) or laziness (ejecting a villain and a hero to get rid of the "problem" easily). TikTok was using this method for a while by ejecting both bullies and victims, and it "solved" the problem but most people see the injustice there. I'm not saying it was bad stewardship, I honestly don't know. I just agree that we shouldn't mak…
>justice
For an individual running a small open source project, there's time enough for coding or detailed justice, but not both. When two parties start pointing fingers and raising hell and its not immediately clear who is in the right, ban both and let them fork it.
Re: GGML – AI at the Edge
#94This is a bang-up idea, you absolutely love to see capital investment on this type of open, commodity-hardware-focused foundational technology. Rock on GGMLers & thank you!
Re: GGML – AI at the Edge
#95Its graph execution is still full of busyloops, e.g.: https://github.com/ggerganov/llama.cpp/blob/44f906e8537fcec9... I wonder how much more efficient it would be when Taskflow lib was used instead, or even inteltbb.
It's not a very good library IMO.
Re: GGML – AI at the Edge
#96Earlier quoted context omitted.
> More details needed to know if this is good stewardship (ejecting two toxic individuals) or laziness (ejecting a villain and a hero to get rid of the "problem" easily). Man, nobody has time for this shit. Leave the games and the drama for the social justice warriors and the furries. People building shit ain't got time for this - ejecting trouble makers is the right way to go regardless of which "side" they're on.
And what do you do when every contributor to the project, including the founder, has been labeled a troublemaker?
Pick the fork that has devs who are focused on contributing code and not pursuing drama.
Re: GGML – AI at the Edge
#97WOW! They are using BFGS! Haven't heard of that in decades! Had to think a little: Yup, the full name is Broyden–Fletcher–Goldfarb–Shanno for iterative unconstrained non-linear optimization!
Some of the earlier descriptions of the optimization being used in the AI learning was about steepest descent, that is, just find the gradient of the function are trying to minimize and move some distance in that direction. Just using the gradient was concerning since that method tends to zig zag where after, say, 100 iterations the distance moved in the 100 iterations might be several times farther than the distance from the starting point of the iterations to the final one. Can visualize this zig zag already in just two dimensions, say, following a river, say, a river that curves, down a valley the river cut over a million years or so, that is, a valley with steep sides. Then gradient descent may keep crossing the river and go maybe 10 feet for each foot downstream!
Right, if just trying to go downhill on a tilted flat plane, then the gradient will point in the steepest descent on the plane and gradient descent will go all way downhill in just one iteration.
In even moderately challenging problems, BFGS can a big improvement.
Re: GGML – AI at the Edge
#98Earlier quoted context omitted.
I wonder if ClosedAI and other companies use the findings of the open source community in their products. For example, do they use QLORA to reduce the costs of training and inference? Do they quantize their models to serve non-subscribing consumers?
Not disagreeing with your points, but saying "ClosedAI" is about as clever as writing M$ for Microsoft back in the day, which is to say not very.
Yeah, I think it feigns meaningful criticism. The "Sleepy Joe"-tier insults are ad-hominem enough that I don't try to respond.
Re: GGML – AI at the Edge
#99Earlier quoted context omitted.
> I also like how when some controversy erupted on the project he just ejected the controversial people and moved on. Good stewardship Do you have more info on the controversy? I'm not sure ejecting developers just because of controversy is honestly good stewardship.
Right. More details needed to know if this is good stewardship (ejecting two toxic individuals) or laziness (ejecting a villain and a hero to get rid of the "problem" easily). TikTok was using this method for a while by ejecting both bullies and victims, and it "solved" the problem but most people see the injustice there. I'm not saying it was bad stewardship, I honestly don't know. I just agree that we shouldn't mak…
> More details needed to know if this is good stewardship (ejecting two toxic individuals) or laziness (ejecting a villain and a hero to get rid of the "problem" easily). TikTok was using this method for a while by ejecting both bullies and victims,
This is SOP for American schools. It's laziness there, since education is supposed to be compulsory. They can't be bothered to investigate (and with today's hostile climate, I don't blame them) so they consign both parties to independent-study programs.
For volunteer projects, throwing both overboard is unfortunate but necessary stewardship. The drama either attracts destabilizes the entire project, which only exists as long as it remains fun for the maintainer. It's tragic, but victims who can't recover gracefully are as toxic as their abusers.
Re: GGML – AI at the Edge
#100This guy is damned good. I sponsored him on Github because his software is dope. I also like how when some controversy erupted on the project he just ejected the controversial people and moved on. Good stewardship. Great code. I recall something like when he first ported it and it worked on my M1 Max he hadn't even yet tested it on Apple Silicon since he didn't have the hardware. Honestly, with this and whisper, I am…
> I also like how when some controversy erupted on the project he just ejected the controversial people and moved on. Good stewardship Do you have more info on the controversy? I'm not sure ejecting developers just because of controversy is honestly good stewardship.
The code is MIT licensed. If you don't agree with the direction the project is taking you can fork it and add whatever you want.
I don't understand why this is so difficult for software developers with GitHub accounts to understand.