and? whats the point of this? Doesn't everyone already know about llama.cpp?
llama.cpp
71–80 of 186 posts
Re: llama.cpp
#72I tried to run in on my Arc A770, but all of the binary releases I could find were compiled without OpenVINO support enabled. I tried compiling it myself, but after two days of the compiler running it failed.
Does the A770 use the Xe driver? If so then it might work with the scripts that I've been using to build llama.cpp with SYCL support for the Arc Pro B70.
https://github.com/cptskippy/battlemage-llm-gateway
It's designed so that you can re-run the scripts to pull the latest updates. When Muse Glimmer was released the other day I just ran the 02 script to build the latest version of llama.cpp with support for it.
Re: llama.cpp
#73Anything that suggests curl into bash just plain sketches me out. (edit: I know, this isn't totally rational, it just seems weird to me. We download and trust a lot of software and run code from a bunch of package repositories as a regular activity...). Git clone llama.cpp and build it, it's not hard. https://github.com/ggml-org/llama.cpp/blob/master/docs/build... literally just a few steps for the basics: git clone…
it's also on the arch repos
Re: llama.cpp
#74Earlier quoted context omitted.
But he’s not using a package, he’s downloading from gir repo and build which exactly same getting a bash script from the same repo and executing it.
It’s not. Try pulling updates after a rebase. Very much not the same as piping arbitrary disembodied text into your shell.
Except the first time you do it
Re: llama.cpp
#75Earlier quoted context omitted.
As an example, webfonts can make rendered text differ from the underlying text that ends up on your clipboard.
Sure, but doesn't this assume that you cannot the publisher anyway? So why would you not trust their homepage but trust their source-code
[dead]
Re: llama.cpp
#76I tried to run in on my Arc A770, but all of the binary releases I could find were compiled without OpenVINO support enabled. I tried compiling it myself, but after two days of the compiler running it failed.
[dead]
Re: llama.cpp
#77> No telemetry Must be tough not to be able to monitor your own models! (The odds that that tagline was AI-generated seem high.)
[dead]
Re: llama.cpp
#78I think I can probably run Gemma 3 12B on my macbook M3 pro with 18GB. The question is, should I do it? This small model is probably not capable of doing a lot or advanced coding or reasoning. What else could it be used for, since it can run locally and privately?
A quantized Qwen3.6-35B-a3b can run in a similar footprint to gemma 12b, but is smarter. It can do coding tasks, if you specify them at a finer-grain than with bigger models.
Re: llama.cpp
#79Earlier quoted context omitted.
does your comment depend on the OS? I thought MLX has better performance on MacOS than llama.cpp
The gap was MUCH larger in the past, but in my tests, oMLX and llama.cpp are now very similar (within 10%) in both prompt processing and generation speed. GGUF ecosystem provides a better selection of quants, in my experience Unsloth ones are excellent.
I thought the main advantage of oMLX is it's less likely to invalidate the KV cache when working with coding agents, which is key when working on a Mac because of the slower prompt processing.
Re: llama.cpp
#80[flagged]