Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple approves driver that lets Nvidia eGPUs work with Arm Macs

twitter.com

Post reply on HN
Print view View original

11–20 of 262 posts

Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs

#11

Post by manmal »

arjie wrote: post #9

Woah, this is exciting. I'm traveling but I have a 5090 lying around at home. I'm eager to give it a go. Docs are here: https://docs.tinygrad.org/tinygpu/ I hope it'll work on an M4 Mac Mini. Does anyone know what hardware to get? You'll need a full ATX PSU to supply power, right? And then tinygrad can do LLM inference on it?

Maybe I’m lacking imagination. But how will a GPU with small-ish but fast VRAM and great compute, augment a Mac with large but slow VRAM and weak compute? The interconnect isn’t powerful enough to change layers on the GPU rapidly, I guess?

Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs

#12

Post by arjie »

manmal wrote: post #11
arjie wrote: post #9

Woah, this is exciting. I'm traveling but I have a 5090 lying around at home. I'm eager to give it a go. Docs are here: https://docs.tinygrad.org/tinygpu/ I hope it'll work on an M4 Mac Mini. Does anyone know what hardware to get? You'll need a full ATX PSU to supply power, right? And then tinygrad can do LLM inference on it?

Maybe I’m lacking imagination. But how will a GPU with small-ish but fast VRAM and great compute, augment a Mac with large but slow VRAM and weak compute? The interconnect isn’t powerful enough to change layers on the GPU rapidly, I guess?

My Mini is actually the smallest model so it actually has "small but slow VRAM" (haha!) so the reason I want the GPU for are the smaller Gemmas or Qwens. Realistically, I'll probably run on an RTX 6000 Pro but this might be fun for home.

Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs

#14

Post by altairprime »

varispeed wrote: post #7
chuckadams wrote: post #5

Earlier quoted context omitted.

Apple got out of the server game long before they adopted aarch64, so that's a trillion worth of server hardware they never would have sold anyway. And probably not actually a trillion.

USD starts sounding more and more like meaningless tokens. Billion here, trillion there. I still have 100 trillion Zimbabwean dollars somewhere.

Feels like that here in the U.S., too.

Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs

#17

Post by embedding-shape »

eoskx wrote: post #8

Interesting, but cannot run CUDA or more to the point `nvidia-smi`.

Well, to be fair, the whole shebang is from a completely different company, that have their own ML library and such, so that isn't that surprising. Although I agree that some CUDA shim or similar would be a lot more interesting, still getting to the place of running inference and training with your very own library is pretty dope already.

Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs

#18

Post by embedding-shape »

brcmthrowaway wrote: post #13

What are the limitations of USB4/Thunderbolt compared with a regular PCIe slot?

Well, for starters, PCIe 5.0 x16 would do something like about 60 GB/s each way, while Thunderbolt 4 does 4 GB/s each way, TB 5 does 8 GB/s each way. If you don't actually hit the bandwidth limits, it obviously matters less. Whether you'd notice a large difference would depends heavily on the type of workload.

Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs

#19

Post by embedding-shape »

frankc wrote: post #15

My main thought is would this allow me to speed up prompt process for large MoE models? That is the real bottleneck for m3ultra. The tokens per second is pretty good.

tinygrad does have pretty neat support for sharding things across various devices relatively easy, that'd help. I'm guessing you'd hit the bandwidth ceiling transferring stuff back and forth though instead.

Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs

#20

Post by lowbloodsugar »

arjie wrote: post #9

Woah, this is exciting. I'm traveling but I have a 5090 lying around at home. I'm eager to give it a go. Docs are here: https://docs.tinygrad.org/tinygpu/ I hope it'll work on an M4 Mac Mini. Does anyone know what hardware to get? You'll need a full ATX PSU to supply power, right? And then tinygrad can do LLM inference on it?

“Lying around”. I’ve got an unopened 5090 in a box that I know will suffer the same fate, so I’m sending it back. So privileged to have the money to impulse buy a 5090 and yet no time to actually do anything with it.
Post reply on HN