Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hands-On with the AMD Ryzen AI Halo

microcenter.com

Post reply on HN
Print view View original

11–20 of 45 posts

Re: Hands-On with the AMD Ryzen AI Halo

#11

Post by daft_pink »

Does anyone else feel like it would be great to be able to purchase $4000 AI box but 128 gigs is not enough. If I spend all that money and it doesn’t really do what I wanted to do, whats the point?

It’s kind of like general aviation where you can go buy a Cessna but it’s only going to realistically get you somewhere you could drive anyways but do you really wanna spend that mush cash to get road trip distance at slightly better than road trip speeds? You really need a 5 million dollar jet and that’s just not practical. That’s sort of how I feel about this device.

Re: Hands-On with the AMD Ryzen AI Halo

#12

Post by prima-facie »

daft_pink wrote: post #11

Does anyone else feel like it would be great to be able to purchase $4000 AI box but 128 gigs is not enough. If I spend all that money and it doesn’t really do what I wanted to do, whats the point? It’s kind of like general aviation where you can go buy a Cessna but it’s only going to realistically get you somewhere you could drive anyways but do you really wanna spend that mush cash to get road trip distance at slig…

The Strix Halo is a great dev machine and a mediocre AI machine. You can run Qwen 3.6 27B at a decent speed, or larger MoE models, and that's about it. For some that's more than enough though, myself included.

Re: Hands-On with the AMD Ryzen AI Halo

#13

Post by wmf »

chmod775 wrote: post #6

This is more of an ad, not a review, and reads like the author has hardly any experience with the things he's trying out. That Z Image Turbo diffusion model would've also run on many consumer GPUs and with way higher performance for a fraction of the price. Misleading.

It's Microcenter. Don't get your reviews from a retailer.

Re: Hands-On with the AMD Ryzen AI Halo

#14

Post by syntaxing »

daft_pink wrote: post #11

Does anyone else feel like it would be great to be able to purchase $4000 AI box but 128 gigs is not enough. If I spend all that money and it doesn’t really do what I wanted to do, whats the point? It’s kind of like general aviation where you can go buy a Cessna but it’s only going to realistically get you somewhere you could drive anyways but do you really wanna spend that mush cash to get road trip distance at slig…

I own one, I don’t feel like the RAM is a huge issue (of course I want 192GB to run something like DS4 Flash). The lack of FP4 and slow memory bandwidth is rough. NVFP4 support is such a huge advantage that I would recommend others to buy a DGX spark over a strix halo if you’re using it purely for AI. Strix halo works better for general computing.

Re: Hands-On with the AMD Ryzen AI Halo

#15

Post by jmyeet »

We're maybe only 2 years away from really useful, relatively affordable local LLM usage. You can buy a 5090 PC for $5-6k but 32GB of VRAM really limits model sizes to ~31B. And that won't change (even with NVidia's next generation) because NVidia uses VARM as an aggressive market segmentation technique.

No, the hope really is these other platforms with a shared memory architecture. The DGX Spark won't be it because of the aforementioned market segmentation. So that leaves two players: AMD and Apple.

The AMD platform is still too low memory bandwidth, currently My great white hope is Apple. They have the market power to get memory and build silicon that coul dhave enough FLOPS to compete with NVidia's platform. They've started talking about it and I've seen rumors they're targeting the M7 generation (2028) for a huge leap. I'll believe it when I see it however.

But the point is, I think we'll be running 31B models at 100+tok/s on enthusiast hardware in 2 years and we'll likely be able to locally run 100-400B models, possibly larger.

Re: Hands-On with the AMD Ryzen AI Halo

#16

Post by prima-facie »

syntaxing wrote: post #14
daft_pink wrote: post #11

Does anyone else feel like it would be great to be able to purchase $4000 AI box but 128 gigs is not enough. If I spend all that money and it doesn’t really do what I wanted to do, whats the point? It’s kind of like general aviation where you can go buy a Cessna but it’s only going to realistically get you somewhere you could drive anyways but do you really wanna spend that mush cash to get road trip distance at slig…

I own one, I don’t feel like the RAM is a huge issue (of course I want 192GB to run something like DS4 Flash). The lack of FP4 and slow memory bandwidth is rough. NVFP4 support is such a huge advantage that I would recommend others to buy a DGX spark over a strix halo if you’re using it purely for AI. Strix halo works better for general computing.

There's a glimmer of hope with ROCmFP4 which seems to double the current throughput: https://github.com/charlie12345/rocmfp4-llama

Re: Hands-On with the AMD Ryzen AI Halo

#17

Post by data-ottawa »

syntaxing wrote: post #9

Highly recommend lemonade server if you have a strix halo desktop. Been using Qwen3.6-35B @ Q_8 as my main driver and it’s been great with 60 TPS for generation. I occasionally use the 27B @ q6 but only get 20-25 TPS for generation with MTP.

I second this.

I used llama swap for a while before leaning into lemonade. The UI has improved a lot, but be careful as the most of the models default to very small 4K context windows by default.

They’re doing some nice things with their Halo models, which load an ensemble of different types of model at the same time. With high vram it’s easy to keep them all in memory, so even though the compute is limited the context switching is fast.

You do lag a bit on the upstream engine releases, the llama.cpp/sd.cpp/whisper libraries are downloaded from inside the app.

vLLM is in experimental mode, I haven’t tested it. It’s limited in the models they suggest, but you can download anything from huggingface with a two click install.

Re: Hands-On with the AMD Ryzen AI Halo

#18

Post by Catloafdev »

syntaxing wrote: post #14
daft_pink wrote: post #11

Does anyone else feel like it would be great to be able to purchase $4000 AI box but 128 gigs is not enough. If I spend all that money and it doesn’t really do what I wanted to do, whats the point? It’s kind of like general aviation where you can go buy a Cessna but it’s only going to realistically get you somewhere you could drive anyways but do you really wanna spend that mush cash to get road trip distance at slig…

I own one, I don’t feel like the RAM is a huge issue (of course I want 192GB to run something like DS4 Flash). The lack of FP4 and slow memory bandwidth is rough. NVFP4 support is such a huge advantage that I would recommend others to buy a DGX spark over a strix halo if you’re using it purely for AI. Strix halo works better for general computing.

Heads up, you can absolutely run DS4 Flash on a 128gb machine - I have it running on my Strix Halo box right now.

https://github.com/antirez/ds4

Re: Hands-On with the AMD Ryzen AI Halo

#19

Post by jmward01 »

It seems like there is a very health space for an MOE targeted GPU where it has essentially an 5070ti with 16gb ish GDDR7 but then also has 128 GB LPDDR5x (or even just DDR5 as expansion dimms on it?). Putting this into the same card would likely reduce the transfer hit when a cache miss happened and the gpu had to load from the slower LPDDR5x. No need to have PCIe 5x16 limiting memory transfer if it is on the card. MOE models could then get near native performance and even models where the active parameters + context didn't quite fit the thrashing would be less of a problem. Not UMA but gets the UMA 'lots of system memory to play with' benefit.

Re: Hands-On with the AMD Ryzen AI Halo

#20

Post by syntaxing »

Catloafdev wrote: post #18
syntaxing wrote: post #14

Earlier quoted context omitted.

I own one, I don’t feel like the RAM is a huge issue (of course I want 192GB to run something like DS4 Flash). The lack of FP4 and slow memory bandwidth is rough. NVFP4 support is such a huge advantage that I would recommend others to buy a DGX spark over a strix halo if you’re using it purely for AI. Strix halo works better for general computing.

Heads up, you can absolutely run DS4 Flash on a 128gb machine - I have it running on my Strix Halo box right now. https://github.com/antirez/ds4

How has it been and how’s the speed? I read online that the it’s ~200 TPS for PP and ~15 TPS for TG. Unfortunately for those speeds, it’s very very hard to use for agentic stuff.
Post reply on HN