Earlier quoted context omitted.
[flagged]
Why are you posting from multiple accounts?
The multiple accounts, the fanatcism. It’s… weird.
llama.cpp
181–190 of 193 posts
Re: llama.cpp
#182llama.cpp works pretty well for me on the Framework 13 laptop, but the current era of "move fast, break things, rarely fix" (sorry, that's how it feels), bites here quite a bit. Two examples: - https://github.com/ggml-org/llama.cpp/pull/25863 Someone's few lines change broke the native (ROCm) support for the AMD GPU inside Framework (and other integrated systems), and any rollback or proper fix is pending for almost…
I have a framework 13, but I couldn't imagine running a local llm on it, how do you do it? Do you have a eGPU?
If you have AMD, you can benefit from the unified memory.
Re: llama.cpp
#183Earlier quoted context omitted.
The first one multiple contributors highlighted the PR as urgent andits had lots of review but it appears to be waiting for another review and/or someone that owns the affected hardware to test that the PR fixes the issue, it wpuld be easy for you to test and report whether or not it does, and the second thing is not related to llama.cpp at all Yes ideally there would be testing every hardware + software combo but th…
I did report my test results on the first one. :) The second I didn't say it's any of llama.cpp's "fault", but it is _related_ to llama.cpp since it's being shipped in another system, aye? Can't stick to the old hash either, because older version have different bugs. E.g. on older versions the same Qwen3.6 model reliably fails to call specific tools due to template issues, while just having the newer llama.cpp versio…
In some cases it may be that llama.cpp causes a bug or lack of functionality downstream but lack of ROCM support appears to be entirely LM Studio’s doing and unrelated to llama.cpp
Sorry if I was too harsh, it’s just that my perception watching the repo has been that the llama.cpp devs are by far the most cautious and slow moving of all the inference implementations, so I found your perspective a bit surprising, I do think that the desire for stable software that never break, and software that supports the latest models and devices/device API’s are conflicting, nothing will do both, and I think that llama.cpp devs do a good job of balancing between shipping features and not breaking users.
Re: llama.cpp
#184Earlier quoted context omitted.
GP is saying "it's not hard", yet you seem to be hearing "you should already know how to do this without ever looking it up". Those aren't the same assertions. It's not a sensible assumption that a process must necessarily be difficult or complex just because you don't already know how to do it. There are an unenumerable number of tasks each of us don't know how to do and have never done before which are not difficul…
Yes, and that's exactly why the curl|sh exists; you don't need to know the weird esoterica of a 26 year old piece of software. Accessibility matters.
>you don't need to know the weird esoterica of a 26 year old piece of software
You really don't need to know much at all in order to compile llama.cpp. You can literally ask any half decent AI agent to do it for you or give you a step by step guide, and troubleshoot any error you get.
Re: llama.cpp
#185[flagged]
Re: llama.cpp
#186Anything that suggests curl into bash just plain sketches me out. (edit: I know, this isn't totally rational, it just seems weird to me. We download and trust a lot of software and run code from a bunch of package repositories as a regular activity...). Git clone llama.cpp and build it, it's not hard. https://github.com/ggml-org/llama.cpp/blob/master/docs/build... literally just a few steps for the basics: git clone…
You can also install it through homebrew https://formulae.brew.sh/formula/llama.cpp
doesnt look like it is the desktop app, its just cli
Re: llama.cpp
#187Earlier quoted context omitted.
You can also install it through homebrew https://formulae.brew.sh/formula/llama.cpp
doesnt look like it is the desktop app, its just cli
brew install --cask llama-app
Re: llama.cpp
#188If I said, “I want a setup that is usable for an agentic coding workflow, and it MUST be local”, what’s the smallest/cheapest option right now? It’s _technically_ possible to get agents running on all kinds of setups but there seems to be an (undefined) floor for useful setups. A lot of the stories people have about getting setups running on relatively low end hardware turn out to have huge compromises or run into is…
It depends on your tolerance level for having less than frontier LLM capabilities. First level worth trying, Qwen 3.6 35B A3B with a 16 GB VRAM (example 1x 5060ti 16gb, 600 USD for the card) with partial GPU offloading. Next level would be Qwen 3.6 27B / new Muse Spark / Gemma 31B with 32 GB VRAM (2x 5060ti or 1x 9700 Pro). Third level would be DeepSeek V4 Flash with 192 GB VRAM (2x Strix Halo at some 8000 USD total). These models can be tried on OpenRuouter etc, or you can deploy vLLM on rented GPUs to get a feel for what level you would want before committing to buying hardware.
Re: llama.cpp
#189Earlier quoted context omitted.
If you revisit my comment and pay attention to the opener: > but you might not be aware that llama-server can do multi-model for a while now you will see that the sentence structure clearly implies both a change compared with a prior state and also lack of any third-party thing. So the answer to the question has already been encoded as text available. _ I can see the desire for explicit validation though. For that, I…
btw, llama-swap provides a nice UI for monitoring performance and logs, and even the ability to stop an infinite session that consumes GPU resources (sometimes that happens). Does the llama.cpp UI provide the same? If not, it is too early to say that llama-swap is “superseded/no longer needed.”
Seconding this. I’ve found that llama-cpp doesn’t completely evict all of the GPU memory with its idle sleep parameter configured.
Re: llama.cpp
#190Earlier quoted context omitted.
I thought the main advantage of oMLX is it's less likely to invalidate the KV cache when working with coding agents, which is key when working on a Mac because of the slower prompt processing.
llama-server also supports saving the kv cache to SSD. I had no issues with cache invalidation using pi.
TIL! When was this functionality added? It wasn’t in llamacpp when I looked in June