Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GAIA – Open-source framework for building AI agents that run on local hardware

amd-gaia.ai

Post reply on HN
Print view View original

11–20 of 43 posts

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#11

Post by ddtaylor »

warwickmcintosh wrote: post #7

ROCm has improved but the reality is you're still fighting the driver stack more than the models. If you're actually doing local inference on AMD you're spending your time on CUDA compatibility layers, not the AI part. Two lines of python is marketing, the gap between demo and working AMD setup is still real.

Ollama works very well in Linux on my AMD hardware. I have a 6800 XT which isn't even originally supported by the ROCm stack in some ways and it "just works" for a ton of very nice models, especially if I seek out quantized versions of the model.

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#12

Post by verdverm »

buryat wrote: post #9
nevi-me wrote: post #6

Earlier quoted context omitted.

Perhaps not a good example, I tried running local models a few times, to much disappointment (actually made me skeptical of LLMs in general for a while). My last experiment in January was trying to run a Qwen model locally (RTX 4080; 128GB RAM; 9950X3D). I must have been doing it extremely wrong because the models that I tried either hallucinated severely or got stuck in a loop. The funniest one was stuck in a "but w…

I have a Mac Studio with 512GB Ram and ran models of different sizes to test out how local agents are and I agree that local models aren't there yet but that depends on whether you need a lot of knowledge or not to answer your question, and I think it should be possible to either distill or train a smaller model that works on a subset of knowledge tailored toward local execution. My main interest is in reducing the l…

I've been running Gemma4, my initial experiments put it around gemini-3-flash levels (vibe evals)

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#13

Post by craftkiller »

wilkystyle wrote: post #5
xrd wrote: post #4

I wanted to believe but anyone who has spent any time trying to run models locally knows this is not going to be solved by two lines of python running on rocm as the example shows.

Curious to hear more. My experience is limited to llama.cpp on Apple silicon so far, but have been eyeing AMD ecosystem from afar.

FWIW I run llama.cpp on AMD hardware using Vulkan. I've got no complaints but also nothing else to compare against.

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#14

Post by h4kunamata »

xrd wrote: post #4

I wanted to believe but anyone who has spent any time trying to run models locally knows this is not going to be solved by two lines of python running on rocm as the example shows.

Not entirely.

I am running OpenWeb UI + Ollama + 7B on a Proxmox LXC container, it consumes less than 2GB, the GPU only has 4GB, and 50% CPU, it is very usable, sometimes faster than online ones to start giving you the answer and 100% offline.

If I replace the GPU with a faster one, I have no need to use online ones.

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#15

Post by sabedevops »

ROCm is finally getting better due to a few well meaning engineers.

But let’s be honest, AMD has been an extremely bad citizen to non-corporate users.

For my iGPU I have to fake GFX900 and build things from source or staging packages to get that working. Support for GFX90c is finally in the pipeline…

The improvements feel like a bodyguard finally letting you through the door just because NVIDIA is eating their lunch and they don’t want their club to be empty.

They strongarm their customers to using “Enterprise” GPUs to be able to play with ROCm, and are only broadening their offerings for market share purposes.

Really shouldn’t reward this behavior.

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#17

Post by androiddrew »

sabedevops wrote: post #15

ROCm is finally getting better due to a few well meaning engineers. But let’s be honest, AMD has been an extremely bad citizen to non-corporate users. For my iGPU I have to fake GFX900 and build things from source or staging packages to get that working. Support for GFX90c is finally in the pipeline… The improvements feel like a bodyguard finally letting you through the door just because NVIDIA is eating their lunch…

Yup, meanwhile Jensen is on the Lexfriedman podcast stating the reason why CUDA is successful is because all thier devices run it. The on ramp is at the individual user.

I have and RDNA4 card and they certainly are prioritizing CDNA over a CDNA + RDNA strategy or a unification strategy.

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#19

Post by coppsilgold »

Nvidia went through a lot of effort to make CUDA operational on their entire lineup, and they did it before deep learning even took off.

You do this thing not because you expect consumers with 5 year old hardware to provide meaningful utilization but as a demo ("let me grab my old gaming machine and do some supercomputing real quick") and a signal that you intend to stay the course. AMD management hasn't realized this even after various Nvidia people said that this was exactly why they did it, at some point the absence of that signal is a signal that the AMD compute ecosystem is an unreliable investment, no?

Re: GAIA – Open-source framework for building AI agents that run on local hardware

#20

Post by cyberax »

xrd wrote: post #4

I wanted to believe but anyone who has spent any time trying to run models locally knows this is not going to be solved by two lines of python running on rocm as the example shows.

Uhmm... I have a local Ollama setup on Linux+AMD, and it was only a bit more involved than this sample. And only because I wanted to run everything in a container.

If you mean that you can't just run the largest unquantized models, then it's indeed true.

Post reply on HN