Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD Ryzen AI Halo – $4k AI Dev Kit

lttlabs.com

Post reply on HN
Print view View original

231–240 of 274 posts

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#231

Post by fodkodrasz »

hypercube33 wrote: post #214
Tenoke wrote: post #63

Earlier quoted context omitted.

I looked for one and it's impossible to find, let alone at a reasonable price + it does suffer from being harder to train/use less common models and workflows (e.g. arbitrary comfyui ones). Spark at least doesnt have that drawback, while AMD has both drawbacks. Waiting for the market to be less insane is somewhat akin to waiting for the s&p500 to drop a decent amount so you can buy in.

Asus rog 13 128gb is still sub $3k last I checked and ticks the boxes. If you get tired of AI it's a kick ass tablet except the weight and can run AAA games for now decently enough.

Is that a notebook? As the thermals don't really compare for a notebook and these high performance minipcs.

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#232

Post by anakaine »

icedchai wrote: post #217
teravor wrote: post #213

Earlier quoted context omitted.

it's worth noting that AMD's software is universally weak and not worth any degree of reliance. and it's not just ROCm, every few months they merge a serious regression into amdgpu and sometimes even backport it into stable. they are amateurs. just a few weeks ago they backported a kernel oops amdgpu null dereference into stable, it's still not fixed.

The "ROCm" situation with Strix Halo was pretty bad for a while. I think it finally stabilized late last year. You needed the right combo of ROCm, Linux kernel, and kernel firmware for it to work reliably. Whenever I rebuild llama.cpp, I wind up using the Vulkan build anyway.

And thats a pretty big annoyance. You need to perfectly line up all the holes in the swiss cheese for the AMD stack to work, then their dev team kicks you in the nuts anyway.

They have made a few attempts at investing in the hardware, but the software side is letting them down hard, and that part is almost entirely their own fault. They have underinvestment in their own stack, and into popular standard and Community libraries that would make it easy to use their gear.

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#233

Post by __rito__ »

kristianp wrote: post #218
__rito__ wrote: post #104

I was in Gray Scott School for HPC last week, and even in scientific usage, CPU-only cases, AMD is still a pain point. Many tools and libraries don't have first class AMD support or any support at all. It loses to Intel in CPU, and NVIDIA in GPU, in case of scientific libraries and HPC-worthy libs, tools. I think people who want an "AI Dev Kit" will lean towards Intel + NVIDIA setup. I am not a fan of Intel, but thei…

> It loses to Intel in CPU Not really, their CPU market share has dropped in the last decade.

I spoke in the context of supercomputer clusters, and usage in scientific HPC.

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#234

Post by rldjbpin »

this product really deserves the "halo" in the name. very difficult to gauge its fit in the current market.

if you want inference, go mac with much higher memory bandwidth. given the price premium here, or the little there is, you might as well.

if you want to finetune and experiment, cuda still has the moat and the kit is not much cheaper, if at all, than dgx spark.

from personal experience, i had access to amd developer cloud with a fair bit of credits. however, even doing inference outside of their supported use cases (which are often dated btw) using vllm was a pain. in the end, despite great computing potential on paper, i decided to not spend more time than its worth on it. if their enterprise cloud continue to have these grievances, i am not optimistic about this kit.

it might be down to skill issue on my end. perhaps if these sell and it gives amd enough motivation to add more software staff in-house, more power to them.

otherwise, good article from labs as usual. nice to know that other kits based on this soc are more or less the same (unsurprisingly).

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#236

Post by phs318u »

chucklenorris wrote: post #224
teravor wrote: post #213

Earlier quoted context omitted.

it's worth noting that AMD's software is universally weak and not worth any degree of reliance. and it's not just ROCm, every few months they merge a serious regression into amdgpu and sometimes even backport it into stable. they are amateurs. just a few weeks ago they backported a kernel oops amdgpu null dereference into stable, it's still not fixed.

biggest mistake was buying an amd laptop. this thing had gpu crashes from day one, and it got worse as time went on. now just playing a video and closing the lid will crash the kernel 100% of the time. amd - never again

Without further info that’s a lot to pin on the bottom layer of the stack. I have an Asus ProArt Ryzen AI HX 390 with 64GB RAM. Running CachyOS and Pop! the thing kept falling over at random times. Running Fedora workstation, it’s rock solid. YMMV.

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#237

Post by prima-facie »

gessha wrote: post #211
prima-facie wrote: post #190

Earlier quoted context omitted.

At the moment, for around $4000-5000 you can either have speed (a GPU + 32GB VRAM), or you can have capacity - a DGX Spark/Halo, but not both. I think once someone comes up with a machine which has both it will easily sell for $10000 and people will be queueing to buy it.

That’s about the price of a workstation-class Nvidia GPU nowadays and if you splurge double, you can get a PCIe version of a data center card.

Sure but where do you stop spending? :)

https://github.com/jamesob/local-llm

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#238

Post by edg5000 »

jaxytee wrote: post #216
codedokode wrote: post #71

Earlier quoted context omitted.

Please note that 8 x 16 Gb DDR4 modules cost $480, and have a bandwidth of 200 Gb/s, the only problem you need 8 DRAM controllers.

Running large MOE models won't saturate the EPYC memory. You'll never see that 200 Gb/s running inference.

Because it'll bottleneck at the CPU? Or because 200 bB/s is overkill when running a model (that fits in memory)?

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#239

Post by nottorp »

Let me guess, the power brick is twice as large as the actual machine and it sounds like a vacuum cleaner because the fans are way too small.

So it's usable only in data center conditions but then why make it in this form factor?

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#240

Post by chrsw »

prima-facie wrote: post #190
Tenoke wrote: post #51

I really want a 128gb+ machine but it's brutal to be at only 256 GB/s for $4k (especially with the drawbacks of both ARM and AMD). I fear that by the time the RTX Spark comes out it'd have to be $6k, and by the time a 128gb or more machine with 700+ GB/s comes out it'd be at $10k, way out of most consumers' hands. Edit: capitalized gb/s to GB/s.

At the moment, for around $4000-5000 you can either have speed (a GPU + 32GB VRAM), or you can have capacity - a DGX Spark/Halo, but not both. I think once someone comes up with a machine which has both it will easily sell for $10000 and people will be queueing to buy it.

It's about $75k https://tinycorp.myshopify.com/products/tinybox-green-v2-wit...
Post reply on HN