I don't know how Apple has evaded regulatory scrutiny for their refusal to sign Nvidia's eGPU drivers since 2018.
Apple approves driver that lets Nvidia eGPUs work with Arm Macs
21–30 of 262 posts
Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs
#22Woah, this is exciting. I'm traveling but I have a 5090 lying around at home. I'm eager to give it a go. Docs are here: https://docs.tinygrad.org/tinygpu/ I hope it'll work on an M4 Mac Mini. Does anyone know what hardware to get? You'll need a full ATX PSU to supply power, right? And then tinygrad can do LLM inference on it?
You can buy a cheap GPU enclosure for about 100$ off ali express.
Takes a standard PSU. However, Mac Minis don't have occulink. So you might be a bit limited by whatever USB C can do.
Now if Intel can get there Arc drivers in order we'll see some real budget fun.
https://www.newegg.com/intel-arc-pro-b70-32gb-graphics-card/...
32 GB of VRAM for 1000$. Plus a 500$ Mac Mini.
Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs
#23Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs
#24What are the limitations of USB4/Thunderbolt compared with a regular PCIe slot?
Well, for starters, PCIe 5.0 x16 would do something like about 60 GB/s each way, while Thunderbolt 4 does 4 GB/s each way, TB 5 does 8 GB/s each way. If you don't actually hit the bandwidth limits, it obviously matters less. Whether you'd notice a large difference would depends heavily on the type of workload.
I think you missed a zero, TB5 does 80GB/s.
Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs
#25Woah, this is exciting. I'm traveling but I have a 5090 lying around at home. I'm eager to give it a go. Docs are here: https://docs.tinygrad.org/tinygpu/ I hope it'll work on an M4 Mac Mini. Does anyone know what hardware to get? You'll need a full ATX PSU to supply power, right? And then tinygrad can do LLM inference on it?
Maybe I’m lacking imagination. But how will a GPU with small-ish but fast VRAM and great compute, augment a Mac with large but slow VRAM and weak compute? The interconnect isn’t powerful enough to change layers on the GPU rapidly, I guess?
> But how will a GPU with small-ish but fast VRAM and great compute, augment a Mac with large but slow VRAM and weak compute?
It would work just like a discrete GPU when doing CPU+GPU inference: you'd run a few shared layers on the discrete GPU and place the rest in unified memory. You'd want to minimize CPU/GPU transfers even more than usual, since a Thunderbolt connection only gives you equivalent throughput to PCIe 4.0 x4.
Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs
#26Earlier quoted context omitted.
Well, for starters, PCIe 5.0 x16 would do something like about 60 GB/s each way, while Thunderbolt 4 does 4 GB/s each way, TB 5 does 8 GB/s each way. If you don't actually hit the bandwidth limits, it obviously matters less. Whether you'd notice a large difference would depends heavily on the type of workload.
I think you missed a zero, TB5 does 80GB/s.
No. It does 80Gbps.
Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs
#27Earlier quoted context omitted.
Well, for starters, PCIe 5.0 x16 would do something like about 60 GB/s each way, while Thunderbolt 4 does 4 GB/s each way, TB 5 does 8 GB/s each way. If you don't actually hit the bandwidth limits, it obviously matters less. Whether you'd notice a large difference would depends heavily on the type of workload.
I think you missed a zero, TB5 does 80GB/s.
No, it does 80 Gb/s. With encoding loss it’s closer to 8GB/s
Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs
#28Woah, this is exciting. I'm traveling but I have a 5090 lying around at home. I'm eager to give it a go. Docs are here: https://docs.tinygrad.org/tinygpu/ I hope it'll work on an M4 Mac Mini. Does anyone know what hardware to get? You'll need a full ATX PSU to supply power, right? And then tinygrad can do LLM inference on it?
Maybe I’m lacking imagination. But how will a GPU with small-ish but fast VRAM and great compute, augment a Mac with large but slow VRAM and weak compute? The interconnect isn’t powerful enough to change layers on the GPU rapidly, I guess?
We've seen many recent projects to stream models direct from SSD to a discrete GPU's limited VRAM on PCs.
How big a bottleneck is Thunderbolt 5 compared to an SSD? Is the 120 Gbps mode only available when linked to a monitor?
Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs
#29Woah, this is exciting. I'm traveling but I have a 5090 lying around at home. I'm eager to give it a go. Docs are here: https://docs.tinygrad.org/tinygpu/ I hope it'll work on an M4 Mac Mini. Does anyone know what hardware to get? You'll need a full ATX PSU to supply power, right? And then tinygrad can do LLM inference on it?
You can buy a cheap GPU enclosure for about 100$ off ali express. Takes a standard PSU. However, Mac Minis don't have occulink. So you might be a bit limited by whatever USB C can do. Now if Intel can get there Arc drivers in order we'll see some real budget fun. https://www.newegg.com/intel-arc-pro-b70-32gb-graphics-card/... 32 GB of VRAM for 1000$. Plus a 500$ Mac Mini.
Those $100 ones don't come with a cage. If you do want a cage, you'll end up with $180 in total, with zero warranty.
Article mentions: "Apple finally approved our driver for both AMD and NVIDIA"
Does not mention Intel (GPUs). Select AMD GPUs work on macOS, but...
Macs (both Intel and ARM) support TB, but eGPU only work on Intel Macs, and basically only with AMD.
Good news is for medium end gaming choices are solid, and CUDA works on AMD these days.
Re: Apple approves driver that lets Nvidia eGPUs work with Arm Macs
#30Earlier quoted context omitted.
Maybe I’m lacking imagination. But how will a GPU with small-ish but fast VRAM and great compute, augment a Mac with large but slow VRAM and weak compute? The interconnect isn’t powerful enough to change layers on the GPU rapidly, I guess?
We've seen many recent projects to stream models direct from SSD to a discrete GPU's limited VRAM on PCs. How big a bottleneck is Thunderbolt 5 compared to an SSD? Is the 120 Gbps mode only available when linked to a monitor?
That’s what, 14GB/s? The GPU‘s VRAM can do 100x that.