Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

jeffgeerling.com

Post reply on HN
Print view View original

221–230 of 236 posts

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#221

Post by MBCook »

ascagnel_ wrote: post #143
re-thc wrote: post #110

Earlier quoted context omitted.

> That they sell to the public? No way. They’ve clearly given up on server stuff and it makes sense for them. Given up is not a given. A lot of the exec team has been changing.

And if the rumors are right -- that hardware SVP John Ternus is next in line for CEO -- I could see a world where the company doubles-down on their specialized hardware vs. services.

They’ve done a dip-in-a-toe thing many times, then gave up.

If I was in charge of a business, and I’m an Apple fan, I wouldn’t touch them. I’d have no faith they’re in it for the long term. I think that would be a common view.

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#222

Post by angoragoats »

wtallis wrote: post #167
angoragoats wrote: post #139

Earlier quoted context omitted.

You’re talking about link aggregation (LACP) here, which requires specific settings on both the switch and client machine to enable, as well as multiple ports on the client machine (in your example, multiple 50Gbps ports). So while it’s likely possible to combine 50Gbps ports like you describe, that’s not what I was referring to.

No, I'm not talking about LACP, I'm talking about configuring four 50Gb links on the switch to operate as a single 200Gb link as if those links were wired up to a single QSFP connector instead of four individual SFP connectors. The switch in question has eight 50Gb ports, and the switch silicon apparently supports configurations that use all of its lanes in groups of four to provide only 200Gb ports. So it might be p…

Ok. I’ve never seen a configuration like this, while using breakout cables to go from higher bandwidth -> multiple lower bandwidth clients is common, so I still disagree with your assertion that it seems “more likely” that this would be supported.

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#223

Post by SV_BubbleTime »

cruffle_duffle wrote: post #193
SV_BubbleTime wrote: post #173

Earlier quoted context omitted.

Look at this guy on his first ram shortage.

Back in the day when 1mb memory sticks ruled the earth there was apparently a memory shortage because some fab burned down or something. Any day now, they’ll fix their shit and ram will be dirt cheap. At least according to my high school buddy. We have always had a ram shortage. We’ve also always been at war with eastasia.

I remember my first 64MB sticks doubled in price after I bought them, I was envied for like 6 months among my friends with their 32MB machines.

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#224

Post by rbanffy »

FuriouslyAdrift wrote: post #220
rbanffy wrote: post #211

Earlier quoted context omitted.

But how long until hardware support is dropped?

I don't know... when is nVidia planning on getting out of the ai business?

When will they discontinue GB10 hardware support because it’s too slow and they want to sell you newer chips?

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#226

Post by ecshafer »

spacedcowboy wrote: post #125
hamdingers wrote: post #27

Earlier quoted context omitted.

> I guess maybe they have some internal M-series server product they just haven’t bothered to release to the public, and features like this are downstream of that? Or do they have some real server-grade product coming down the line, and are releasing this ahead of it so that 3rd party software supports it on launch day?

I worked on some of the internal server hardware. Yes they do have their own boards. Apple used to be all-in on Linux, but the newer chips are far and away more power-efficient, and power is one of the (if not the ) major cost of outfitting a datacenter, at least over time. These machines are very much internal - you can cram a lot of M-series (to use the public nomenclature) chips onto a rack-sized PCB. I was never…

Are these internal servers full of M-series chips running a server max osx build then as well?

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#227

Post by yonatan8070 »

Tepix wrote: post #128

Linux already has RDMA support but it cannot yet use Thunderbolt. It's probably quite a bit of work to add everything that's required. Is anyone working on it? It would be great to have this for those cheap Strix Halo boxes with 128GB quad channel DDR5-8000 for using two or three of them with their 2 USB4 ports (which are Thunderbolt capable) to fit larger models.

I assume that's Thunderbolt 5? From my experience, eGPUs over USB 4/TB3 work just fine (from a technical point of view, in practice 40Gbps isn't enough BW and performance is shit)

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#228

Post by Tepix »

yonatan8070 wrote: post #227
Tepix wrote: post #128

Linux already has RDMA support but it cannot yet use Thunderbolt. It's probably quite a bit of work to add everything that's required. Is anyone working on it? It would be great to have this for those cheap Strix Halo boxes with 128GB quad channel DDR5-8000 for using two or three of them with their 2 USB4 ports (which are Thunderbolt capable) to fit larger models.

I assume that's Thunderbolt 5? From my experience, eGPUs over USB 4/TB3 work just fine (from a technical point of view, in practice 40Gbps isn't enough BW and performance is shit)

Yes, that works but what's needed here is remotely accessing RAM on another PC via RDMA via Thunderbolt. Not accessing a standalone eGPU.

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#229

Post by nottorp »

gflarity wrote: post #200
nottorp wrote: post #179

Earlier quoted context omitted.

> Electricity is cheap. American :)

Or Canadian... off peak in Ontario is <$0.03 US if you prioritize off peak...

Hate you from Europe :)

[roughly 23 us cents / kWh on my last bill]

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#230

Post by spacedcowboy »

ecshafer wrote: post #226
spacedcowboy wrote: post #125

Earlier quoted context omitted.

I worked on some of the internal server hardware. Yes they do have their own boards. Apple used to be all-in on Linux, but the newer chips are far and away more power-efficient, and power is one of the (if not the ) major cost of outfitting a datacenter, at least over time. These machines are very much internal - you can cram a lot of M-series (to use the public nomenclature) chips onto a rack-sized PCB. I was never…

Are these internal servers full of M-series chips running a server max osx build then as well?

Apple's OS builds are a lot more flexible than most people give them credit for. That's why essentially the same OS scales from a watch to a Mac Pro. You can mix and match the ingredients of the OS for a given device pretty much at will, as long as the dependencies are satisfied. And since you own the OS, dependencies are often configurable.
Post reply on HN