Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD Ryzen AI Halo – $4k AI Dev Kit

lttlabs.com

Post reply on HN
Print view View original

251–260 of 274 posts

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#251

Post by iterateoften »

freedomben wrote: post #241
phs318u wrote: post #236

Earlier quoted context omitted.

Without further info that’s a lot to pin on the bottom layer of the stack. I have an Asus ProArt Ryzen AI HX 390 with 64GB RAM. Running CachyOS and Pop! the thing kept falling over at random times. Running Fedora workstation, it’s rock solid. YMMV.

Same, I have an all AMD framework laptop 16 with the gaming GPU, and running fedora, and it has been rock solid. I really couldn't ask for more

Just to add another data point, my AMD laptop was as the poster above described, unreliable drivers, glitches and crashes.

Personally I’ll never buy AMD again.

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#253

Post by verdverm »

SwellJoe wrote: post #199
my123 wrote: post #187

Earlier quoted context omitted.

No the Spark uses UEFI + ACPI Regular Ubuntu or Fedora ISOs do work out of the box Just needs the Nvidia GPU driver install afterwards (And the realtek 10gbe module oot, or blocklist if not used)

Ah, that makes all the difference. Being normal has a lot of value. I mostly take it back. I still don't like the custom Ubuntu Nvidia ships, but if the system is normal any Linux distro for ARM works, it becomes an entirely viable thing.

Nvidia didn't make a great out-of-box experience for sure, one of the first things I do on my Spark is set init 3 and disable bluetooth. Docker was working on one, not the second. Many typical local tools used in Ai were not installed (ollama,vllm,llama.cpp... all MIA)

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#254

Post by verdverm »

wolttam wrote: post #153
roadside_picnic wrote: post #123

Earlier quoted context omitted.

What's the tokens/sec you're getting on that setup (genuinely curious because it's a setup I haven't actually run myself)?

Using regular DeepSeek-V4-Flash, I see 2000tok/s prompt processing and from 40 to 50 tok/s generation. Performance doesn't drop much at long contexts, DSv4 is really nice for that. With DeepSeek-V4-Flash-DSpark (Deepseek's new speculative decoding scheme), which is still barely supported anywhere, we're seeing a more steady 45-55 tok/s with bursts into the 60s.

What quant size are you using? Just picked up my second spark and a cable and wanted to try this one out for fun. I generally run a bunch of models for different purposes (easy / bulk tasks) and use APIs for harder tasks.

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#256

Post by verdverm »

nightski wrote: post #108
cyanydeez wrote: post #48

Earlier quoted context omitted.

yeah, so $500 spread https://www.microcenter.com/product/699008/nvidia-dgx-spark is what the current price appears to be. The differences are basically, sparks require ARM and sparks allow interconnects; so if you do have dreams of electric sheep to chain them together, you're not gonna get the AMD halo units. But if you just want to putz around with a dev machine and do other things, not sure you'd want a spark.

They had it on sale last week for $3999, it will likely happen again. Also if you are willing to buy ASUS/Acer/MSI you can get them cheaper, in the same range as well. Those units are identical (mainboard/ram/chipset/connectivity), they only tend to differ in SSD being offered.

The more beneficial difference between DGX and OEM Spark is in cooling. DGX has had cooling issue based on user reports.

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#257

Post by cmrdporcupine »

verdverm wrote: post #253
SwellJoe wrote: post #199

Earlier quoted context omitted.

Ah, that makes all the difference. Being normal has a lot of value. I mostly take it back. I still don't like the custom Ubuntu Nvidia ships, but if the system is normal any Linux distro for ARM works, it becomes an entirely viable thing.

Nvidia didn't make a great out-of-box experience for sure, one of the first things I do on my Spark is set init 3 and disable bluetooth. Docker was working on one, not the second. Many typical local tools used in Ai were not installed (ollama,vllm,llama.cpp... all MIA)

I mean, I never expected to have pre-packaged versions of any of those things that wouldn't just fall out of sync with upstream immediately anyways?

I basically treated it as a stock Ubuntu machine but on Aarch64, and ignored the stuff they installed. And then went looking for docker images for vllm that made sense and were up to date and hopefully tuned for NVFP4 on the hardware and...

... that was more the disappointing part.

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#259

Post by verdverm »

cmrdporcupine wrote: post #257
verdverm wrote: post #253

Earlier quoted context omitted.

Nvidia didn't make a great out-of-box experience for sure, one of the first things I do on my Spark is set init 3 and disable bluetooth. Docker was working on one, not the second. Many typical local tools used in Ai were not installed (ollama,vllm,llama.cpp... all MIA)

I mean, I never expected to have pre-packaged versions of any of those things that wouldn't just fall out of sync with upstream immediately anyways? I basically treated it as a stock Ubuntu machine but on Aarch64, and ignored the stuff they installed. And then went looking for docker images for vllm that made sense and were up to date and hopefully tuned for NVFP4 on the hardware and... ... that was more the disappoi…

The Spark does a system update on first boot, so they would have updated had they been installed

I'm running vllm + GoModel on k3s, works like a charm. Even wrote some CUE last night to generate the values file for my Helm chart. It calculates the GPU percentage for me from the memGB I assign to a model.

Post reply on HN