Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Lemonade by AMD: a fast and open source local LLM server using GPU and NPU

lemonade-server.ai

Post reply on HN
Print view View original

41–50 of 133 posts

Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU

#41

Post by RealFloridaMan »

nijave wrote: post #2

Anyone compare to ollama? I had good success with latest ollama with ROCm 7.4 on 9070 XT a few days ago

It is optimized for compatibility across different APIs as well as has specific hardware builds for AMD GPUs and NPUs. It’s run by AMD.

Under the hood they are both running llama.cpp, but this has specific builds for different GPUs. Not sure if the 9070 is one, I am running it on a 370 and 395 APU.

Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU

#42

Post by metalliqaz »

my most powerful system is Ryzen+Radeon, so if there are tools that do all the hard work of making AI tools work well on my hardware, I'm all for it. I find it very frustrating to get LLMs, diffusion, etc. working fast on AMD. It's way too much work.

Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU

#43

Post by metalliqaz »

parsimo2010 wrote: post #39
moconnor wrote: post #13

Is... is this named because they have a lemon they're trying to make the most of?

I think saying "L-L-M" sounds kind of like "lemon," so this is an LLM-aid (sounds like lemonade).

so obvious and yet I didn't connect the dots. thank you

Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU

#44

Post by RealFloridaMan »

ilaksh wrote: post #18

Cool but is there a reason they can't just make PRs for vLLM and llama.cpp? Or have their own forks if they take too long to merge?

They use the latest llama.cpp under the hood but built for specific AMD GPU hardware.

Lemonade is really just a management plane/proxy. It translates ollama/anthropic APIs to OpenAI format for llama.cpp. It runs different backends for sst/tts and image generation. Lets you manage it all in one place.

Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU

#47

Post by hombre_fatal »

gnarlouse wrote: post #46

Maybe it's a language barrier problem, but "by AMD" makes me think its a project distributed by AMD. Is that actually the case? I'm not seeing any reason to believe it is.

> You can reach us by filing an issue, emailing lemonade@amd.com

Found this on the github readme.

Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU

#48

Post by buildbot »

gnarlouse wrote: post #46

Maybe it's a language barrier problem, but "by AMD" makes me think its a project distributed by AMD. Is that actually the case? I'm not seeing any reason to believe it is.

It’s a community project supported and sponsored by AMD according to their GitHub; https://github.com/lemonade-sdk/lemonade

AMD employees work on it/have been making blog posts about it for a bit.

Re: Lemonade by AMD: a fast and open source local LLM server using GPU and NPU

#49

Post by guipsp »

gnarlouse wrote: post #46

Maybe it's a language barrier problem, but "by AMD" makes me think its a project distributed by AMD. Is that actually the case? I'm not seeing any reason to believe it is.

It is mostly developed by AMD and used to be hosted on the AMD github iirc
Post reply on HN