Hands-On with the AMD Ryzen AI Halo
microcenter.com
Hands-On with the AMD Ryzen AI Halo
1–10 of 45 posts
Re: Hands-On with the AMD Ryzen AI Halo
#2Until RAM prices drop and can economically get machines with 256GB, 512GB and higher bandwidth... I frankly think the local AI story is going to be still fairly muted for most people.
My Spark can do Qwen3.6 MoE A3B at 60 to 70-ish token/second and that's really good, but there's limits the usefulness of that model. It's not useful for coding, in any case.
Once people can run something like GLM 5.2 at lower quants (512GB could do a passable job), then I think the story changes.
Whether we ever see DRAM as cheap as it was ever again, I don't know.
Re: Hands-On with the AMD Ryzen AI Halo
#3Unfortunately, the table of models and tokens per second (TPS) and time to first token (TTFT) is not helpful without specifying the quantization of the model.
Re: Hands-On with the AMD Ryzen AI Halo
#4Until RAM prices drop and can economically get machines with 256GB, 512GB and higher bandwidth... I frankly think the local AI story is going to be still fairly muted for most people. My Spark can do Qwen3.6 MoE A3B at 60 to 70-ish token/second and that's really good, but there's limits the usefulness of that model. It's not useful for coding, in any case. Once people can run something like GLM 5.2 at lower quants (5…
Agree - the 128GB Strix Halo is capable if you use LLMs as assistants, but it's not so good if you use LLMs as agents (or worse, agent teams/swarms) since all of the models that can fit on it are pretty dumb compared to frontier or near-frontier models. You can at best hope for Sonnet-level capabilities.
That doesn't mean that local models are useless though! If Mythos/Sol is an ASI that threatens to take your job and turn you into paperclips, then Qwen/Gemma is an old-fashioned office secretary that loyally helps you with tasks but doesn't have a good grasp of details. Every white-collar worker 50 years ago would have killed to have a hard-working personal secretary.
Re: Hands-On with the AMD Ryzen AI Halo
#5Until RAM prices drop and can economically get machines with 256GB, 512GB and higher bandwidth... I frankly think the local AI story is going to be still fairly muted for most people. My Spark can do Qwen3.6 MoE A3B at 60 to 70-ish token/second and that's really good, but there's limits the usefulness of that model. It's not useful for coding, in any case. Once people can run something like GLM 5.2 at lower quants (5…
Part of me wonders, would 3d xpoint (if still around) be a viable option?
Yeah it is slower than real RAM by a good amount for latency, but you can get similar bandwidth and the cost was history about half of the same size DDR.
Re: Hands-On with the AMD Ryzen AI Halo
#6This is more of an ad, not a review, and reads like the author has hardly any experience with the things he's trying out. That Z Image Turbo diffusion model would've also run on many consumer GPUs and with way higher performance for a fraction of the price. Misleading.
Re: Hands-On with the AMD Ryzen AI Halo
#7Until RAM prices drop and can economically get machines with 256GB, 512GB and higher bandwidth... I frankly think the local AI story is going to be still fairly muted for most people. My Spark can do Qwen3.6 MoE A3B at 60 to 70-ish token/second and that's really good, but there's limits the usefulness of that model. It's not useful for coding, in any case. Once people can run something like GLM 5.2 at lower quants (5…
Part of me wonders, would 3d xpoint (if still around) be a viable option? Yeah it is slower than real RAM by a good amount for latency, but you can get similar bandwidth and the cost was history about half of the same size DDR.
Possibly - I've heard anecdotal reports that old Intel Optane chips are in hot demand right now. Intel/Micron probably would have made a killing if they had kept that product line alive for a few more years. Never miss an attempt to snatch defeat from the jaws of victory!
Re: Hands-On with the AMD Ryzen AI Halo
#8Until RAM prices drop and can economically get machines with 256GB, 512GB and higher bandwidth... I frankly think the local AI story is going to be still fairly muted for most people. My Spark can do Qwen3.6 MoE A3B at 60 to 70-ish token/second and that's really good, but there's limits the usefulness of that model. It's not useful for coding, in any case. Once people can run something like GLM 5.2 at lower quants (5…
Part of me wonders, would 3d xpoint (if still around) be a viable option? Yeah it is slower than real RAM by a good amount for latency, but you can get similar bandwidth and the cost was history about half of the same size DDR.
I was thought experimenting the other day... ~10 nVME drives striped and running parallel could approximate the memory bandwidth of DDR5 DRAM in a box like this. Like you say, latency wouldn't compete but on raw throughput would be comparable.
Not anymore cost effective, I guess, but gets you the ability to work over very large model sizes maybe. But the problem is that tensor matmul etc hardware wouldn't work effectively with it.
Useful for KVCache though.
Re: Hands-On with the AMD Ryzen AI Halo
#9Highly recommend lemonade server if you have a strix halo desktop. Been using Qwen3.6-35B @ Q_8 as my main driver and it’s been great with 60 TPS for generation. I occasionally use the 27B @ q6 but only get 20-25 TPS for generation with MTP.
Re: Hands-On with the AMD Ryzen AI Halo
#10seems like an absolute amateur wrote this article