Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GAIA – Open-source framework for building AI agents that run on local hardware

amd-gaia.ai

Post reply on HN
Print view View original

21–30 of 43 posts

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#21

Post by androiddrew »

coppsilgold wrote: post #19

Nvidia went through a lot of effort to make CUDA operational on their entire lineup, and they did it before deep learning even took off. You do this thing not because you expect consumers with 5 year old hardware to provide meaningful utilization but as a demo ("let me grab my old gaming machine and do some supercomputing real quick") and a signal that you intend to stay the course. AMD management hasn't realized thi…

You got it right I think. I’m sitting with two “AI Ready Radeon AI Pro 9700 workstation cards, which are RDNA4 not CDNA. My experience is that my cards are not a priority. Individual engineers at AMD may care, the company doesn’t. I have been trying since February to get ahold of anyone responsible for shipping tuned Tensile gfx1201 kernels in rocm-libs, which is used by Ollama.its been three weeks since I raised enough hell on the discord to get a response, but they still can’t find “who” is responsible for Tensile tuning, and “if” they are even going to do it for the gfx12* cards.

Don’t get me started with vLLM and AITER.

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#22

Post by lostmsu »

nevi-me wrote: post #6
wilkystyle wrote: post #5

Earlier quoted context omitted.

Curious to hear more. My experience is limited to llama.cpp on Apple silicon so far, but have been eyeing AMD ecosystem from afar.

Perhaps not a good example, I tried running local models a few times, to much disappointment (actually made me skeptical of LLMs in general for a while). My last experiment in January was trying to run a Qwen model locally (RTX 4080; 128GB RAM; 9950X3D). I must have been doing it extremely wrong because the models that I tried either hallucinated severely or got stuck in a loop. The funniest one was stuck in a "but w…

I hope you are not running models under Q8, preferably Q8 directly from the vendor.

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#23

Post by sandworm101 »

xrd wrote: post #4

I wanted to believe but anyone who has spent any time trying to run models locally knows this is not going to be solved by two lines of python running on rocm as the example shows.

I am running q 4xgpu rig at home (similar to a mining rig) doing everything from llms to content creation. I have learned a lot. Having an AI rig today is much like having an early PC in the 80s. You dont appeciate the possible uses until you have it in your hands.

All you need is a used GPU slapped onto any disused ddr4 mobo. New 5060s, the 16gb models, can do basically everything now.

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#24

Post by Shitty-kitty »

sabedevops wrote: post #15

ROCm is finally getting better due to a few well meaning engineers. But let’s be honest, AMD has been an extremely bad citizen to non-corporate users. For my iGPU I have to fake GFX900 and build things from source or staging packages to get that working. Support for GFX90c is finally in the pipeline… The improvements feel like a bodyguard finally letting you through the door just because NVIDIA is eating their lunch…

The problem is the split CDNA/RDNA architecture. A problem they are adressing with their upcoming unified UDMA.

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#25

Post by robwwilliams »

buryat wrote: post #9
nevi-me wrote: post #6

Earlier quoted context omitted.

Perhaps not a good example, I tried running local models a few times, to much disappointment (actually made me skeptical of LLMs in general for a while). My last experiment in January was trying to run a Qwen model locally (RTX 4080; 128GB RAM; 9950X3D). I must have been doing it extremely wrong because the models that I tried either hallucinated severely or got stuck in a loop. The funniest one was stuck in a "but w…

I have a Mac Studio with 512GB Ram and ran models of different sizes to test out how local agents are and I agree that local models aren't there yet but that depends on whether you need a lot of knowledge or not to answer your question, and I think it should be possible to either distill or train a smaller model that works on a subset of knowledge tailored toward local execution. My main interest is in reducing the l…

Also running gemma-4 on Apple M5 Max. As fast or faster than Opus 4.6 extended but not of course the same competence. However, great tunability with llama.cpp and no issues related to IP leakage.

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#29

Post by musicale »

buryat wrote: post #9
nevi-me wrote: post #6

Earlier quoted context omitted.

Perhaps not a good example, I tried running local models a few times, to much disappointment (actually made me skeptical of LLMs in general for a while). My last experiment in January was trying to run a Qwen model locally (RTX 4080; 128GB RAM; 9950X3D). I must have been doing it extremely wrong because the models that I tried either hallucinated severely or got stuck in a loop. The funniest one was stuck in a "but w…

I have a Mac Studio with 512GB Ram and ran models of different sizes to test out how local agents are and I agree that local models aren't there yet but that depends on whether you need a lot of knowledge or not to answer your question, and I think it should be possible to either distill or train a smaller model that works on a subset of knowledge tailored toward local execution. My main interest is in reducing the l…

> Mac Studio with 512GB Ram

Nice to score one of those.

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#30

Post by wingtw »

sandworm101 wrote: post #23
xrd wrote: post #4

I wanted to believe but anyone who has spent any time trying to run models locally knows this is not going to be solved by two lines of python running on rocm as the example shows.

I am running q 4xgpu rig at home (similar to a mining rig) doing everything from llms to content creation. I have learned a lot. Having an AI rig today is much like having an early PC in the 80s. You dont appeciate the possible uses until you have it in your hands. All you need is a used GPU slapped onto any disused ddr4 mobo. New 5060s, the 16gb models, can do basically everything now.

Can you specify a bit, what gpus and how do you wire them "together"? Nvlink?
Post reply on HN