Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD Strix Halo RDMA Cluster Setup Guide

github.com

Post reply on HN
Print view View original

61–70 of 90 posts

Re: AMD Strix Halo RDMA Cluster Setup Guide

#61

Post by Tuna-Fish »

jmyeet wrote: post #44
Hikikomori wrote: post #35

Earlier quoted context omitted.

What is simplex ethernet?

Imagine two computers A and B. A has two NICs, A1 and A2. B has B1 and B2. So 4 NICs total. You connect directly A1 to B1 and A2 to B2 with crossover cables. You then route all the traffic from A to B over A1 to B1 and all the traffic from B to A over B2 to A2. Why do you do all this? To avoid collisions and the loss of effective bandwidth from back-offs. It only really works with 2 computers because if you add a 3rd…

That's not how modern ethernet works at all. A single NIC talking directly to an another one has no collisions ever. Depending on what your channel is, either you have separate wires for the directions, or you are using a hybrid circuit (as in telegraphs, the term is so overloaded it's hard to google). Either way, packets going in one direction never wait for packets going in the other.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#62

Post by somewhatrandom9 »

barbacoa wrote: post #37
sspiff wrote: post #32

Earlier quoted context omitted.

I had a Strix Halo laptop with 128GB which unfortunately died last week. I paid 2800 euro for it. If I buy the same machine today, the sticker price is 7899. The device was not perfect by any means, but the ability to run fairly large models is some kind of magic.

>sticker price is 7899. It's not even worth it at that point. You can get a used enterprise grade SXM baseboard with 4-8 V100/A100 GPUs off eBay at a similar price. That will even get you actual HMB ram and NVlink. Along with 10x the AI performance, assuming you don't care about your electricity bill of course.

You can get a new M5 Max MacBook pro with 128 GB unified ram (targeted by Antirez for DwarfStar4) even after the Apple price increases, it's less than 7899 by at least $1000. And you probably won't pull more than 100 Watts.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#64

Post by fulafel »

fulafel wrote: post #42
kamranjon wrote: post #34

Earlier quoted context omitted.

Ah yea after watching one of the creators youtube videos I realize these benchmarks are combining prefill and decode which isn't super helpful - it seems this struggles with the exact same bottlenecks as all strix halo setups, memory bandwidth. It seems this is still significantly slower than equivalent memory sizing on Mac hardware.

How are the memory bandwidths specs of Macbooks vs this?

I looked it up: 512 GB/s for the two node AMD cluster, Macbook Pro with M5 CPU has 153 GB/s. But you can get faster Macs with M5 Pro or M5 Max.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#65

Post by dehugger »

somewhatrandom9 wrote: post #62
barbacoa wrote: post #37

Earlier quoted context omitted.

>sticker price is 7899. It's not even worth it at that point. You can get a used enterprise grade SXM baseboard with 4-8 V100/A100 GPUs off eBay at a similar price. That will even get you actual HMB ram and NVlink. Along with 10x the AI performance, assuming you don't care about your electricity bill of course.

You can get a new M5 Max MacBook pro with 128 GB unified ram (targeted by Antirez for DwarfStar4) even after the Apple price increases, it's less than 7899 by at least $1000. And you probably won't pull more than 100 Watts.

I have one of these. Got it a few weeks before the price increases. On the 14" version charging is limited to 96 watts, but the chip can pull north of that with adequate cooling, so the battery will literally drain while plugged in.

It isn't a problem for me, more amusing than anything else (I run in Low Power mode 90% of the time) but worth knowing for anyone thats thinking about pushing the hardware to its limit 24/7.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#66

Post by omgwtfbyobbq »

gruez wrote: post #5
pixelpoet wrote: post #3

I have two 128gb Strix Halos and have been extremely excited about Antirez's (Redis author) work on DS4, especially with 4bit quant using two machines: https://github.com/antirez/ds4 Right now the speed isn't good for GLM 5.2, Deepseek V4 Flash speed is okay for me (actually reading the output) and quite usable. See kyuz0's great recent video here: https://www.youtube.com/watch?v=PkKXm_mKCCM With a bit more speed and…

>The biggest problem is all the tech companies making consumer hardware completely unaffordable, and I don't think this is accidental. Look at Micron's profits and share price lately... You realize "tech companies" isn't a monolith? Micron charging inflated prices doesn't magically benefit OpenAI. The "high prices keep out competitors" theory doesn't make much sense either. It's like saying Dennys benefits from highe…

Dennys can benefit from higher egg prices if they can lock in long term contracts with suppliers for lower egg prices when smaller companies selling directly to consumers can't.

I think that realistically, companies compete against each other as individuals and compete against smaller companies and individuals acting more like cartels/monopolies, and that's what OP is referring to in terms of hardware purchasing/contacts/pricing. This also extends outside of tech to investing, so it's likely not just tech responsible for this.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#67

Post by sspiff »

somewhatrandom9 wrote: post #62
barbacoa wrote: post #37

Earlier quoted context omitted.

>sticker price is 7899. It's not even worth it at that point. You can get a used enterprise grade SXM baseboard with 4-8 V100/A100 GPUs off eBay at a similar price. That will even get you actual HMB ram and NVlink. Along with 10x the AI performance, assuming you don't care about your electricity bill of course.

You can get a new M5 Max MacBook pro with 128 GB unified ram (targeted by Antirez for DwarfStar4) even after the Apple price increases, it's less than 7899 by at least $1000. And you probably won't pull more than 100 Watts.

You are comparing US pricing with EU pricing. EU pricing includes 21% VAT and currency conversion "rounding up".

The cheapest 128GB Macbook Pro here costs €7.949,00.

No doubt a better value than the HP, and will depreciate a lot less quickly, but just as expensive. Unfortunately, not being able to run Linux is a breaking point for me.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#68

Post by sspiff »

barbacoa wrote: post #37
sspiff wrote: post #32

Earlier quoted context omitted.

I had a Strix Halo laptop with 128GB which unfortunately died last week. I paid 2800 euro for it. If I buy the same machine today, the sticker price is 7899. The device was not perfect by any means, but the ability to run fairly large models is some kind of magic.

>sticker price is 7899. It's not even worth it at that point. You can get a used enterprise grade SXM baseboard with 4-8 V100/A100 GPUs off eBay at a similar price. That will even get you actual HMB ram and NVlink. Along with 10x the AI performance, assuming you don't care about your electricity bill of course.

Yeah it isn't worth it, but comparing a server with a laptop is also not a relevant comparison.

I didn't get a Strix Halo laptop because it was the best bang per buck, I got it because it was an awesome machine that could do a little bit of everything, fit in a backpack and only needed 140W.

But noone should buy one at 7899, obviously. It was a tough sell for me at the old 2800 pricing.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#69

Post by kamranjon »

fulafel wrote: post #64
fulafel wrote: post #42

Earlier quoted context omitted.

How are the memory bandwidths specs of Macbooks vs this?

I looked it up: 512 GB/s for the two node AMD cluster, Macbook Pro with M5 CPU has 153 GB/s. But you can get faster Macs with M5 Pro or M5 Max.

Strix halo I believe is 256GB/s max memory bandwidth and M5 Max is 614GB/s - M3 Ultra is up to 800gb/s

Re: AMD Strix Halo RDMA Cluster Setup Guide

#70

Post by nicman23 »

pixelpoet wrote: post #3

I have two 128gb Strix Halos and have been extremely excited about Antirez's (Redis author) work on DS4, especially with 4bit quant using two machines: https://github.com/antirez/ds4 Right now the speed isn't good for GLM 5.2, Deepseek V4 Flash speed is okay for me (actually reading the output) and quite usable. See kyuz0's great recent video here: https://www.youtube.com/watch?v=PkKXm_mKCCM With a bit more speed and…

if strix was 2k now i would insta buy 1
Post reply on HN