A fun observation: pulling models sends ~200mbit of progress updates to your browser
Lemonade by AMD: a fast and open source local LLM server using GPU and NPU
91–100 of 133 posts
Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU
#92Is... is this named because they have a lemon they're trying to make the most of?
I exclusively buy AMD hardware for local inference. For open drivers, power efficiency, and cost AMD beats Nvidia easily for consumers.
Any recommendations in the current market? Love how plug and play and is on Linux from the driver side of things.
Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU
#93[flagged]
> Been running local LLMs on my 7900 XTX for months and the ROCm experience has been... rough.
Just out of curiosity... how so?
I only ask because I've been running local models (using Ollama) on my RX 7900 XTX for the last year and a half or so and haven't had a single problem that was ROCm specific that I can think of. Actually, I've barely had any problems at all, other than the card being limited to 24GB of VRAM. :-(
I'm halfway tempted to splurge on a Radeon Pro board to get more VRAM, but ... haven't bitten the bullet yet.
Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU
#94Earlier quoted context omitted.
Have you used it with any agents or claw? If so, which model do you run?
As another data point. Running Qwen3.5 122B at 35t/s as a daily driver using Vulcan llama.cpp on kernel 7.0.0rc5 on a Framework Desktop board (Strix Halo 128). Also a pair of AMD AI Pro r9700 cards as my workhorses for zimageturbo, qwen tts/asr and other accessory functions and experiments. Finally have a Radeon 6900 XT running qwen3.5 32B at 60+t/s for a fast all arounder. If I buy anything nvidia it will be only fo…
Are the dedicated GPU cards on another machine or you’re using eGPU with the framework?
Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU
#95Earlier quoted context omitted.
Have you used it with any agents or claw? If so, which model do you run?
As another data point. Running Qwen3.5 122B at 35t/s as a daily driver using Vulcan llama.cpp on kernel 7.0.0rc5 on a Framework Desktop board (Strix Halo 128). Also a pair of AMD AI Pro r9700 cards as my workhorses for zimageturbo, qwen tts/asr and other accessory functions and experiments. Finally have a Radeon 6900 XT running qwen3.5 32B at 60+t/s for a fast all arounder. If I buy anything nvidia it will be only fo…
How is the performance for Z-Image on the R9700s?
Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU
#96I have been using lemonade for nearly a year already. On Strix Halo I am using nothing else - although kyuz0's toolboxes are also nice ( https://kyuz0.github.io/amd-strix-halo-toolboxes/ ) Nowadays you get TTS, STT, text & image generation and image editing should also be possible. Besides being able to run via rocm, vulkan or on CPU, GPU and NPU. Quite a lot of options. They have a quite good and pragmatic pace in d…
How much of a speedup might I get for, say, Qwen3.5-122B if I were to run with lemonade on my Strix Halo vs running it using vulkan with llama.cpp ?
You would get similar performance. Lemonade is designed as a turnkey (optimized for AMD Hardware) for local AI models. The software helps you manage backends (llama.cpp, flm, whispercpp, stable‑diffusion.cpp, etc) for different GenAI modalities from a single utility.
On the performance side, lemonade comes bundled with ROCm and Vulkan. These are sourced from https://github.com/lemonade-sdk/llamacpp-rocm and https://github.com/ggml-org/llama.cpp/releases respectively.
Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU
#97Earlier quoted context omitted.
I have two Strix Halo devices at hand. Privately a framework desktop with 128gb and at work 64GB HP notebook. The 64GB machine can load Qwen3.5 30B-A3B, with VSCode it needs a bit of initial prompt processing to initialize all those tools I guess. But the model is fighting with the other resources that I need. So I am not really using it anymore these days, but I want to experiment on my home machine with it. I just…
Qwen3-Coder-Next works well on my 128GB Framework Desktop. It seems better at coding Python than Qwen3.5 35B-A3B, and it's not too much slower (43 tg/s compared to 55 tg/s at Q4). 27B is supposed to be really good but it's so slow I gave up on it (11-12 tg/s at Q4).
The 8 bit MLX unsloth quant of qwen3-coder-next seems to be a local best on an MBB M5 Max with 128GB memory. With oMLX doing prompt caching I can run two in parallel doing different tasks pretty reasonably. I found that lower quants tend to lose the plot after about 170k tokens in context.
Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU
#98What is the lowest process I can implement this on?
Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU
#99[dead]
Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU
#100Is... is this named because they have a lemon they're trying to make the most of?
I exclusively buy AMD hardware for local inference. For open drivers, power efficiency, and cost AMD beats Nvidia easily for consumers.
You have got to be joking.
My three NVIDIA cards are more power efficient than my one AMD card, both at idle and during usage.
Official ROCm is like pulling teeth with poor support for desktop cards. Debian, a volunteer led project, have better ROCm CI than AMD and support more cards.
Look at any benchmarks. NV midrange cards are faster than AMD and at least a generation in front. Owning a 7900XTX is an embarrassing disappointment.
I like AMD and want them to succeed, but they are way behind NV in this area.