Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD Strix Halo RDMA Cluster Setup Guide

github.com

Post reply on HN
Print view View original

51–60 of 90 posts

Re: AMD Strix Halo RDMA Cluster Setup Guide

#51

Post by Keyframe »

sspiff wrote: post #32
pixelpoet wrote: post #3

I have two 128gb Strix Halos and have been extremely excited about Antirez's (Redis author) work on DS4, especially with 4bit quant using two machines: https://github.com/antirez/ds4 Right now the speed isn't good for GLM 5.2, Deepseek V4 Flash speed is okay for me (actually reading the output) and quite usable. See kyuz0's great recent video here: https://www.youtube.com/watch?v=PkKXm_mKCCM With a bit more speed and…

I had a Strix Halo laptop with 128GB which unfortunately died last week. I paid 2800 euro for it. If I buy the same machine today, the sticker price is 7899. The device was not perfect by any means, but the ability to run fairly large models is some kind of magic.

everything strix halo went 2-3x bananas, same ballpark figures as apple hardware now and lead times on all of those are in months. Ridiculous where we ended up at.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#52

Post by Keyframe »

barbacoa wrote: post #37
sspiff wrote: post #32

Earlier quoted context omitted.

I had a Strix Halo laptop with 128GB which unfortunately died last week. I paid 2800 euro for it. If I buy the same machine today, the sticker price is 7899. The device was not perfect by any means, but the ability to run fairly large models is some kind of magic.

>sticker price is 7899. It's not even worth it at that point. You can get a used enterprise grade SXM baseboard with 4-8 V100/A100 GPUs off eBay at a similar price. That will even get you actual HMB ram and NVlink. Along with 10x the AI performance, assuming you don't care about your electricity bill of course.

ignoring the fact one would need a bit of a different setup (chassis, PSU) to run it, I casually looked and there's nothing below $25-50k euros for such a board decked out, depending on the config. TBH even that doesn't sound bad, but I wouldn't even know where to start how to run it.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#53

Post by skinfaxi »

barbacoa wrote: post #37
sspiff wrote: post #32

Earlier quoted context omitted.

I had a Strix Halo laptop with 128GB which unfortunately died last week. I paid 2800 euro for it. If I buy the same machine today, the sticker price is 7899. The device was not perfect by any means, but the ability to run fairly large models is some kind of magic.

>sticker price is 7899. It's not even worth it at that point. You can get a used enterprise grade SXM baseboard with 4-8 V100/A100 GPUs off eBay at a similar price. That will even get you actual HMB ram and NVlink. Along with 10x the AI performance, assuming you don't care about your electricity bill of course.

Can you link because 8k is the price of two A100s so that would be a steal.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#54

Post by justsomehnguy »

pixelpoet wrote: post #23
rjzzleep wrote: post #21

Earlier quoted context omitted.

Last year you could buy a AI Max 395+ with 128G for 2.5k, now it's almost $4k. Or maybe you're right, I originally remembered 2k as well. I wanted to wait for the AI Max 395+ upgrade of my laptop, and now it makes no sense to upgrade.

I think people buying laptops for AI use are, sorry, just plain crazy. You overpay for the screen and keyboard and battery and whatever, plus you get much worse thermal performance because of basic physics (area vs volume). My Framework Desktop has a Noctua cooler which works really well. [Tangent: all my life I've been downvoted into a smoking hole in the ground, particularly on reddit r/hardware, for questioning th…

Maybe there is a reason?

It's like you are advocating for a public transport instead of a personal car but when questioned how to get to a place which is not erviced by a public network your solution is to rent a bus.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#56

Post by barbacoa »

skinfaxi wrote: post #53
barbacoa wrote: post #37

Earlier quoted context omitted.

>sticker price is 7899. It's not even worth it at that point. You can get a used enterprise grade SXM baseboard with 4-8 V100/A100 GPUs off eBay at a similar price. That will even get you actual HMB ram and NVlink. Along with 10x the AI performance, assuming you don't care about your electricity bill of course.

Can you link because 8k is the price of two A100s so that would be a steal.

4x A100s with the baseboard for $9k (USD).

https://www.ebay.com/itm/336632412718

Buy one and let me use it when you're sleeping.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#57

Post by barbacoa »

Keyframe wrote: post #52
barbacoa wrote: post #37

Earlier quoted context omitted.

>sticker price is 7899. It's not even worth it at that point. You can get a used enterprise grade SXM baseboard with 4-8 V100/A100 GPUs off eBay at a similar price. That will even get you actual HMB ram and NVlink. Along with 10x the AI performance, assuming you don't care about your electricity bill of course.

ignoring the fact one would need a bit of a different setup (chassis, PSU) to run it, I casually looked and there's nothing below $25-50k euros for such a board decked out, depending on the config. TBH even that doesn't sound bad, but I wouldn't even know where to start how to run it.

This is from the USA site:

https://www.ebay.com/itm/157742745616

It's always a gamble buying used electric but per the description, fully decked out server with 256gb vram.

Though at 2U it is going to sound like a 747 taking off from your office.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#58

Post by codingrightnow »

sdf4j wrote: post #11
gruez wrote: post #5

Earlier quoted context omitted.

>The biggest problem is all the tech companies making consumer hardware completely unaffordable, and I don't think this is accidental. Look at Micron's profits and share price lately... You realize "tech companies" isn't a monolith? Micron charging inflated prices doesn't magically benefit OpenAI. The "high prices keep out competitors" theory doesn't make much sense either. It's like saying Dennys benefits from highe…

You got it wrong. Use appliances instead of eggs. If getting an oven gets more expensive I rather keep going to Dennys. It’s classic capex vs opex. I’d keep paying my openai subscription instead of dropping $3k to run a subpar model. If the thing costs $1k I would consider it.

The big AI companies already have all of the cheap hardware to run on. It's more like if Denny's bought up all of the eggs when prices were normal, held onto them, then kept buying to keep egg prices up.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#59

Post by gessha »

barbacoa wrote: post #57
Keyframe wrote: post #52

Earlier quoted context omitted.

ignoring the fact one would need a bit of a different setup (chassis, PSU) to run it, I casually looked and there's nothing below $25-50k euros for such a board decked out, depending on the config. TBH even that doesn't sound bad, but I wouldn't even know where to start how to run it.

This is from the USA site: https://www.ebay.com/itm/157742745616 It's always a gamble buying used electric but per the description, fully decked out server with 256gb vram. Though at 2U it is going to sound like a 747 taking off from your office.

So that’s where all the used V100s are going. I’ve been thinking about making a water cooled version of those because I don’t have a rack to put these servers on.

Re: AMD Strix Halo RDMA Cluster Setup Guide

#60

Post by gruez »

codingrightnow wrote: post #58
sdf4j wrote: post #11

Earlier quoted context omitted.

You got it wrong. Use appliances instead of eggs. If getting an oven gets more expensive I rather keep going to Dennys. It’s classic capex vs opex. I’d keep paying my openai subscription instead of dropping $3k to run a subpar model. If the thing costs $1k I would consider it.

The big AI companies already have all of the cheap hardware to run on. It's more like if Denny's bought up all of the eggs when prices were normal, held onto them, then kept buying to keep egg prices up.

>The big AI companies already have all of the cheap hardware to run on.

Have they? Aren't they doing a massive datacenter build-out right now? Moreover the massive profits for Micron and Nvidia must be coming from somewhere, and I doubt it's price-sensitive consumers.

Post reply on HN