Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

jeffgeerling.com

Post reply on HN
Print view View original

201–210 of 236 posts

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#201

Post by pudquick »

e28eta wrote: post #199

> For example: did you know there's no way to run a system upgrade (like to 26.2) via SSH I did not know this. I thought the `softwareupdate` command was built for this use case, and thought it worked over ssh. It sure looks like it should work, but I don’t have a mac I can try it on right now.

He's wrong, it's possible. It's just that root privileges alone is insufficient due to how the signing on LocalPolicy works on M series Macs

https://support.apple.com/guide/security/contents-a-localpol...

The manpage for the command provides information on credential usage on Apple Silicon devices.

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#202

Post by Dylan16807 »

pdpi wrote: post #44
zozbot234 wrote: post #14

Earlier quoted context omitted.

> Neural accelerators to get prompt prefill time down. Apple Neural Engine is a thing already, with support for multiply-accumulate on INT8 and FP16. AI inference frameworks need to add support for it. > this setup can support up to 4 Mac devices because each Mac must be connected to every other Mac!! Do you really need a fully connected mesh? Doesn't Thunderbolt just show up as a network connection that RDMA is ran…

> Do you really need a fully connected mesh? Doesn't Thunderbolt just show up as a network connection that RDMA is ran on top of? If you daisy chain four nodes, then traffic between nodes #1 and #4 eat up all of nodes #2 and #3's bandwidth, and you eat a big latency penalty. So, absent a switch, the fully connected mesh is the only way to have fast access to all the memory.

Obviously don't daisy chain, that wastes ports so badly. But if you connect 4 nodes into a loop, it goes fine. Relaying only adds 33% extra traffic. And what specifically are the latency numbers you have in mind?

If you have 3 links per box, then you can set up 8 nodes with a max distance of 2 hops and an average distance of 1.57 hops. That's not too bad. It's pretty close to having 2 links each to a big switch.

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#203

Post by FuriouslyAdrift »

burnte wrote: post #189
SV_BubbleTime wrote: post #173

Earlier quoted context omitted.

Look at this guy on his first ram shortage.

I've been in this game so long, seen so many shortages that I'm not even worried. Right now prices are high, manufacturers are switching production, and in 6 months there's going to be a glut of supply. It's all a long game, folks. Play it long.

It takes about a decade (and $10s of billions) to bring new fabs online

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#204

Post by FuriouslyAdrift »

rbanffy wrote: post #147
doctorpangloss wrote: post #114

Earlier quoted context omitted.

the GB10 is only the same performance as a 3090. gb10 uses way less power. i'm not sure why anyone would buy a mac studio instead of a gb10 machine for this use case.

> i'm not sure why anyone would buy a mac studio instead of a gb10 For an AI-only use case, the GB10s make sense, but they are only OK as desktop workstations, and I’m not sure for how long DGX OS will be updated, as dedicated AI machines have somewhat short lives. Apple computers, OTOH, have much longer lives, and desktops live the longest. I retired my Mac Mini a year after the machine was no longer getting OS upda…

DGX OS (based on Ubuntu) is used for all of nVidias GPU compute systems so it's probably going to be around for a while.

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#205

Post by buildbot »

sgjohnson wrote: post #52
angoragoats wrote: post #49

Earlier quoted context omitted.

As with most 40+GbE ports, the 400Gbit ports can be split into 2x200Gbit ports with the use of special cables. So you can connect a total of 6 machines at 200Gbit.

Breakout cables typically split to 4. e.g. QSFP28 (100GbE) splits into 4x SFP28s (25GbE each), because QSFP28 is just 4 lanes of SFP28. Same goes for QSFP112 (400GbE). Splits into SFP112s. It’s OSFP that can be split in half, i.e. into QSFPs.

This is incorrect - they can split however the driving chip supports. (Q|O)SFP(28|56|112|+) can all be split to a single differential lane. All (Q|O)SFP(28|56|112|+) does is provide basically direct, high quality links to whatever you chips SERDES interfaces can do. It doesn't even have to be ethernet/IB data - I have a SFP module that has a SATA port lol.

There's also splitting at the module level, for example I have a PCIe card that is actually a fully self hosted 6 port 100GB switch with it's own onboard Atom management processor. The card only has 2 MPO fiber connectors - but each has 12 fibers, which each can carry 25Gbps. You need a special fiber breakout cable but you can mix anywhere between 6 100GbE ports and 24 25Gbe ports.

https://www.silicom-usa.com/pr/server-adapters/switch-on-nic...

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#206

Post by wmf »

throwaway2037 wrote: post #123
wtallis wrote: post #53

Earlier quoted context omitted.

Ah, good point. Though if splitter cables are an option, then it seems more likely that the 50G ports could be combined into a 200G cable. Marvell's product brief for that switch chip does say it's capable of operating as an 8x 200G or 4x 400G switch, but Mikrotik may need to do something on their end to enable that configuration.

I'm not trolling here: Do you think that Marvell sells the chips wholesale buy the vendor buys the feature set (IP/drivers/whatever)? That would allow Marvell to effectively sell the same silicon but segment the market depending upon what buyers needs. Example: A buyer might need a config that is just a bunch of 50GB/s ports and another 100GB/s ports and another a mix. (I'm thinking about blowing fuses in the manuf p…

The Marvell 98DX7335 switch ASIC has 32 lanes that can be configured any way the vendor wants. There aren't any fuses and it can even be reconfigured at runtime (e.g. a 400G port can be split into 2x200G).

Are the smaller 98DX7325 and 98DX7321 the same chip with fuses blown? I wouldn't be surprised.

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#207

Post by pjmlp »

raw_anon_1111 wrote: post #187
pjmlp wrote: post #183

Earlier quoted context omitted.

As someone living in a country, Portugal, where Apple had a single reseller, Interlog. I could count with my hand fingers how many Macs I have seen being used between being born in the 70's and 2000's, up to 10. My university graduation project was porting a visualisation framework from NeXTSTEP into Windows, because already there the university could not see a future with NeXT. The fact that people believe Apple's c…

At no point was Microsoft spending more on Office for Mac than they were making on selling the Mac version. It cost some. But famously Microsoft used byte code for Office that was portable and was dog slow around Office 5. And it’s not my “believing”, it’s math. Apple lost far more than the net $150 million before it became popular. This isn’t my reading the history books. My first computer was an Apple //e in 1986 a…

The fact is Microsoft did spent R&D money to support Apple customers, with a product without which, Apple would even be less relevant in the late 90's.

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#208

Post by zhengyi13 »

ddalex wrote: post #166
RunSet wrote: post #159

Earlier quoted context omitted.

AI bubble will do wonders for used RAM prices when it pops.

There is no popping. We cannot have enough compute for the forseeable future.

"The market can stay irrational longer than you can stay solvent"

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#209

Post by raw_anon_1111 »

pjmlp wrote: post #207
raw_anon_1111 wrote: post #187

Earlier quoted context omitted.

At no point was Microsoft spending more on Office for Mac than they were making on selling the Mac version. It cost some. But famously Microsoft used byte code for Office that was portable and was dog slow around Office 5. And it’s not my “believing”, it’s math. Apple lost far more than the net $150 million before it became popular. This isn’t my reading the history books. My first computer was an Apple //e in 1986 a…

The fact is Microsoft did spent R&D money to support Apple customers, with a product without which, Apple would even be less relevant in the late 90's.

They didn’t do it out of charity. They did it for profit. Microsoft was on the stage when the Mac was introduced in 1984. I don’t think they were ever seriously considering stopping Mac development. They ported office to PPC Macs in 1995 before the joint announcement 2 years later.

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#210

Post by jeffbee »

Asmod4n wrote: post #194
jeffbee wrote: post #70

Earlier quoted context omitted.

I honestly forgot they still made the Mac Pro. Amazing that they have these ready to ship on their website. But at a 50% premium over similar but faster Mac Studio models, what is the point? You can't usefully put GPUs in them as far as I know. You'd have to have a different PCIe need to make it make sense.

all PCIe lanes combined in that machine can do over 1 terabit. Would be quite the networking beast.

The M2 Ultra has 32 off-world PCIe lanes, 8 of which are obligated to the SSDs. That leaves only 24 lanes for the 7 slots. That's 8 times less than you'd get from an EPYC, which is the kind of thing a normal user would put in a rack if they did not need to use macos.
Post reply on HN