Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD Ryzen AI Halo – $4k AI Dev Kit

lttlabs.com

Post reply on HN
Print view View original

131–140 of 274 posts

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#131

Post by c7b »

ahmadyan wrote: post #117
p1esk wrote: post #113

Earlier quoted context omitted.

where do you see "twice the memory bandwidth"?

He is making things up. It is the same bandwidth as DGX Spark (256 GB/s vs 273 GB/s) and far behind M3 Ultra (~819 GB/s)

You're the one making things up. An M3 Ultra with 128GB RAM doesn't exist, the M3 Max has 410GB/s bandwidth [0]. I was of course talking about the M4 Max with 546GB/s, which was closer to twice the price of a Strix Halo mini PC in a typical configuration when it was still available. And memory bandwidth isn't everything, NVidia's lead in software is substantial, look up any tests comparing them side-by-side.

[0] https://en.wikipedia.org/wiki/Apple_M3 [1] https://en.wikipedia.org/wiki/Apple_M4

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#132

Post by alexdns »

mhitza wrote: post #19
alexdns wrote: post #4

Bosgame is $2799 does the same thing if you plan to run only 1 of them

The repeated claim that all these different forms are not directly comparable is a very strange aspect. Only thing that separates them is the build quality and the extra 20W of boost the framework desktop and this variant support. They have a note on the thermals but no measurement of noise. Doesn't matter if it's stricly a whoosh or a whine, only if they bother people in the same room. And the small ones like Bosgam…

The framework has a noisy PSU , the bosgame is noisy However you can fix both - the bosgame leaves you a large margin to do so

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#133

Post by adrian_b »

c7b wrote: post #107

This is just a little under the price of NVidia's DGX Spark with CUDA or a Mac with 128GB and twice the memory bandwidth. The point of Strix Halo used to be that it was half the price of those way more capable machines. You'd be crazy to buy the AMD chip at this price. But the hardware market is generally crazy right now, so I'm sure this will sell as well, unfortunately.

Only the top variants of M5 Max have more than twice the memory bandwidth of AMD, but those are much more expensive, i.e. over $10,000.

M5 Pro has as slightly higher memory bandwidth, but currently it is available only with up to 64 GB of DRAM. Even with this small memory, it is more expensive than either AMD or NVIDIA, especially when you do not want a puny SSD, but a normal-sized SSD, i.e. big enough to put there an LLM if you want to compute a quantization yourself (i.e. more than $5600 with a 4 TB SSD & 64 GB DRAM).

If you want to do LLM inference, I do not see Apple as a competitive solution, as their price is much, much higher, while also having limited expansion for SSDs, where you need a lot of space if you want to store a few LLMs.

The only thing that is correct is that the AMD Strix Halo system used to be much cheaper than NVIDIA, but now it has the same price. The CPU of Strix Halo is better than that of NVIDIA, but the NVIDIA GPU is likely to be better than the AMD GPU and CUDA is guaranteed to work fine.

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#134

Post by hadlock »

dabinat wrote: post #62
Tenoke wrote: post #51

I really want a 128gb+ machine but it's brutal to be at only 256 GB/s for $4k (especially with the drawbacks of both ARM and AMD). I fear that by the time the RTX Spark comes out it'd have to be $6k, and by the time a 128gb or more machine with 700+ GB/s comes out it'd be at $10k, way out of most consumers' hands. Edit: capitalized gb/s to GB/s.

A Mac Studio is a much better buy in terms of memory bandwidth, but impossible to buy in a 128 GB configuration. Honestly there aren’t great options right now and it’s probably better to wait for the market to be less insane.

Apple rumor mill is suggesting that we may see M5 Mac Studio announced in September at that event. Apple just increased their pricing across the board, so I'm not holding my breath that they will be reasonably priced. They also cancelled development of their M6 in favor of the future M7 which I suspect will be a major, AI-focused upgrade compared to the M4->M5 upgrade

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#135

Post by icedchai »

codedokode wrote: post #125
bradfa wrote: post #81

Earlier quoted context omitted.

Each memory controller interface is a not-insignificant number of PCB traces. Increasing the number of memory controllers may dramatically increase the number of PCB layers (or may not, it really depends on the CPU pinout) but it definitely will increase the number of pins on the CPU socket. This is one of the main reasons (the other is the number of PCIe lanes) why high end desktop and server CPUs have like double t…

Then what's about using 4-8 cheapest motherboards with 64Gb DDR4 and a cheap CPU, and connecting them via PCIE x16 sockets? And as for DRAM channels, typical cheap motherboard has 2 channels and 4 slots, it should not be super difficult to add 2 more channels.

Don't you need a new CPU to support more channels? It doesn't sound like a cheap, trivial change to upgrade my Ryzen to a Threadripper.

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#136

Post by grubbs »

frugalmail wrote: post #93
grubbs wrote: post #24

Earlier quoted context omitted.

Cheapest I've been able to get a DGX Spark FE is now around $4700 just FYI. This is from multiple vendors in higher-ed.

I'm looking at Amazon and I see $4k GB10 devices right now (not an affiliate link) https://www.amazon.com/ASUS-Supercomputer-Superchip-Supports...

Yea I mentioned the FE version which is 4TB NVME. The one you linked is 1TB NVME.

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#137

Post by SwellJoe »

I have a Strix Halo device, and like it, but at this price, might as well buy the Nvidia-based ASUS GX 10, if you're buying it for AI. CUDA remains the stronger ecosystem. The AMD is a better desktop machine, as it has a better CPU, but the Nvidia will be a little faster and a little better supported for inference and training workloads. You can almost always do the same things with ROCm, but you're going to work a little more.

Though, I will say that Nvidia ships a dogshit custom Ubuntu on their hardware that's hard to deal with. Nvidia is not good at software. I keep thinking they'll get better at it, but I've been dealing with their Jetson line for a couple of years now, and it still sucks. Still a clumsy custom Ubuntu, and it's not as easy as simply installing a different Linux version as it's a complicated image-based thing and no UEFI. At least, I assume they ship Ubuntu on the big devices; I've only dealt with the little embedded Jetson machines. The AMD stuff, being a regular x86_64 PC, you can install pretty much any Linux. I immediately put Fedora on mine.

Re: AMD Ryzen AI Halo – $4k AI Dev Kit

#140

Post by adrian_b »

Catloafdev wrote: post #23

These devices were great when they were cheaper than the DGX Spark. But when they cost the same price (unless the Spark has shot up too), there's no reason to buy this over a Spark. The Spark is literally a faster version of this, with better software support. Edit: And I say that as an owner of a Ryzen AI Max 395 device.

Depends on what you use it for.

The CPU of Ryzen is better than that of DGX Spark, especially for modern programs that have been updated to use AVX-512 (i.e. it has a significantly higher multithreaded performance).

Only for GPU applications the NVIDIA system is likely to be better.

Post reply on HN