Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

jeffgeerling.com

Post reply on HN
Print view View original

231–236 of 236 posts

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#231

Post by spacedcowboy »

mcculley wrote: post #157
spacedcowboy wrote: post #124

Earlier quoted context omitted.

Considering that Apple is moving away from Linux in the datacenter to its own devices, I'm not sure that's the case. The apple machines aren't available to the consumer (they're rack-mounted, dozens of chips per PCB board, custom-made machines) but they're much less power-hungry, just as fast (or more so), much cheaper for them to make rather than buy, and natively support their own ecosystem. Some of the machine-des…

> dozens of chips per PCB board Have there been leaks or something about these internal machines? I am curious to know more.

No idea. I worked on them so I didn't really need the leak :)

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#232

Post by g947o »

pjmlp wrote: post #134

In an ideal world, Apple would have released a Mac Pro with card slots for doing this kind of stuff. Instead we get gimmicks over Thunderbolt.

I can imagine Apple shipping Mac Pros with add-ons that allows running local inference with minimal setups. "Look, just spend $50k on this machine and you get a usable LLM server that can be shared for a team." But they don't seem particularly interested in that market.

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#233

Post by g947o »

pjmlp wrote: post #137
jauntywundrkind wrote: post #90

I really hope AMD or Intel can get on the clue train and respond. Intel in particular has half a decade of having extremely amazing Thunderbolt ports on their mobile chips, built in (alas not present on desktop chips, for shame). There's been not bad but not great thunderbolt host-to-host networking, that TCP can go over, but the system to system connectivity had been a total afterthought, not at all tuned for obviou…

In the real world, you get a desktop PC with a bunch of GPUs connected on the same bus talking to each other. No need for multiple computers talking over thunderbolt.

probably need to build your own PC. This is not really possible on most prebuilt gaming PCs.

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#234

Post by lostmsu »

FuriouslyAdrift wrote: post #220
rbanffy wrote: post #211

Earlier quoted context omitted.

But how long until hardware support is dropped?

I don't know... when is nVidia planning on getting out of the ai business?

Yeah, if you had experience with their Jetson boards, you'd know Nvidia is not well regarded for their OS support.

TK1 support stopped after under 4 years. Basically they released it with some version of Ubuntu (14.04 LTS) and never upgraded it.

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#235

Post by sandworm101 »

nottorp wrote: post #229
gflarity wrote: post #200

Earlier quoted context omitted.

Or Canadian... off peak in Ontario is <$0.03 US if you prioritize off peak...

Hate you from Europe :) [roughly 23 us cents / kWh on my last bill]

https://www.oeb.ca/consumer-information-and-protection/elect...

"On-Peak/Weekdays 4 p.m. – 9 p.m. = 39.1c"

Re: 1.5 TB of VRAM on Mac Studio – RDMA over Thunderbolt 5

#236

Post by burnte »

Dylan16807 wrote: post #218
burnte wrote: post #217

Earlier quoted context omitted.

You've answered the question! They're redirecting those chips to industrial use which makes desktop products more expensive and less available. Samsung is also extending DDR4 production, for example.

I thought you were listing a switch in production that would relieve the shortages after we wait a few months. Switching away from desktop memory makes the shortages worse. So why do you expect there to be a glut in 6 months? If you meant glut of memory suitable for datacenter GPUs, I don't expect that nearly so soon. That market can absorb extra chips pretty easily unless we see a really harsh pop really soon.

The same chips go into desktops as servers, so this is just redirecting the chips to another assembly line. I think there's a good chance the memory market will see a huge boost in supply to server spaces, and some memory will switch back to desktop use.
Post reply on HN