GAIA – Open-source framework for building AI agents that run on local hardware
1–10 of 43 posts
Re: GAIA – Open-source framework for building AI agents that run on local hardware
#2Not so clear from their page but from
https://www.tipranks.com/news/amd-stock-slips-despite-a-majo...
I read:
" In addition to that, the update allows these agents to be turned into desktop apps for multiple operating systems. "
This seems like a new way to create app: Create an (AI) app that creates apps.
Re: GAIA – Open-source framework for building AI agents that run on local hardware
#3In case you are interested:
Requirement Minimum
Processor AMD Ryzen AI 300-series
Re: GAIA – Open-source framework for building AI agents that run on local hardware
#4I wanted to believe but anyone who has spent any time trying to run models locally knows this is not going to be solved by two lines of python running on rocm as the example shows.
Re: GAIA – Open-source framework for building AI agents that run on local hardware
#5I wanted to believe but anyone who has spent any time trying to run models locally knows this is not going to be solved by two lines of python running on rocm as the example shows.
Curious to hear more. My experience is limited to llama.cpp on Apple silicon so far, but have been eyeing AMD ecosystem from afar.
Re: GAIA – Open-source framework for building AI agents that run on local hardware
#6I wanted to believe but anyone who has spent any time trying to run models locally knows this is not going to be solved by two lines of python running on rocm as the example shows.
Curious to hear more. My experience is limited to llama.cpp on Apple silicon so far, but have been eyeing AMD ecosystem from afar.
Perhaps not a good example, I tried running local models a few times, to much disappointment (actually made me skeptical of LLMs in general for a while).
My last experiment in January was trying to run a Qwen model locally (RTX 4080; 128GB RAM; 9950X3D). I must have been doing it extremely wrong because the models that I tried either hallucinated severely or got stuck in a loop. The funniest one was stuck in a "but wait, ..." loop.
I fortunately had started experimenting with Claude, so I opted to pay Anthropic more money for tokens (work already covers the bill, this was for personal use).
That whole experience + a noisy GPU, put me off the idea of running/building local agents.
Re: GAIA – Open-source framework for building AI agents that run on local hardware
#7ROCm has improved but the reality is you're still fighting the driver stack more than the models. If you're actually doing local inference on AMD you're spending your time on CUDA compatibility layers, not the AI part. Two lines of python is marketing, the gap between demo and working AMD setup is still real.
Re: GAIA – Open-source framework for building AI agents that run on local hardware
#8Not so clear from their page but from https://www.tipranks.com/news/amd-stock-slips-despite-a-majo... I read: " In addition to that, the update allows these agents to be turned into desktop apps for multiple operating systems. " This seems like a new way to create app: Create an (AI) app that creates apps.
The stock is peaking highs again and people are quoting articles saying "stock slips"...
Re: GAIA – Open-source framework for building AI agents that run on local hardware
#9Earlier quoted context omitted.
Curious to hear more. My experience is limited to llama.cpp on Apple silicon so far, but have been eyeing AMD ecosystem from afar.
Perhaps not a good example, I tried running local models a few times, to much disappointment (actually made me skeptical of LLMs in general for a while). My last experiment in January was trying to run a Qwen model locally (RTX 4080; 128GB RAM; 9950X3D). I must have been doing it extremely wrong because the models that I tried either hallucinated severely or got stuck in a loop. The funniest one was stuck in a "but w…
I have a Mac Studio with 512GB Ram and ran models of different sizes to test out how local agents are and I agree that local models aren't there yet but that depends on whether you need a lot of knowledge or not to answer your question, and I think it should be possible to either distill or train a smaller model that works on a subset of knowledge tailored toward local execution. My main interest is in reducing the latency and it feels that the local agents that work at high speeds should be an answer to this but it's not something that someone is trying to solve yet. Feels like if I could get a smaller model that could run at incredible speed locally that could unlock some interesting autoresearching.
Re: GAIA – Open-source framework for building AI agents that run on local hardware
#10I wanted to believe but anyone who has spent any time trying to run models locally knows this is not going to be solved by two lines of python running on rocm as the example shows.
Curious to hear more. My experience is limited to llama.cpp on Apple silicon so far, but have been eyeing AMD ecosystem from afar.
The main thing to consider is that how you run the models does not need to be coupled to the what you send models (and how you orchestrate agents).
I've used several agent frameworks and they all support many different providers from cloud to local. These are orthogonal responsibilities. I'm using VertexAI for cloud and ollama on a minisforum with rocm locally. There is a dropdown to change between them.