Apple need to sort out their software.
Mac OS is a horrible experience.
Apple discontinues the Mac Pro
251–260 of 672 posts
Re: Apple discontinues the Mac Pro
#252Earlier quoted context omitted.
But those Thunderbolt links are slower than modern PCIe. If there's actually a M5-based Mac Studio with the same Thunderbolt support, you'll be better off e.g. for LLM inference, streaming read-only model weights from storage as we've seen with recent experiments than pushing the same amount of data via Thunderbolt. It's only if you want to go beyond local memory constraints (e.g. larger contexts) that the Thunderbol…
Wasn't streaming models from storage into limited memory a case where it was impressive that you could make the elephant dance at all? If you want to get usable speeds from very large models that haven't been quantitized to death on local machines, RDMA over Thunderbolt enables that use case. Consumer PC GPUs don't have enough RAM, enterprise GPUs that can handle the load very well are obscenely expensive, Strix Halo…
The bad performance you saw was with very limited memory and very large models, so streaming weights from storage was a huge bottleneck. If you gradually increase RAM, more and more of the weights are cached and the speed improves quite a bit, at least until you're running huge contexts and most of the RAM ends up being devoted to that. Is the overall speed "usable"? That's highly subjective, but with local inference it's convenient to run 24x7 and rely on non-interactive use. Of course scaling out via RDMA on Thunderbolt is still there as an option, it's just not the first approach you'd try.
Re: Apple discontinues the Mac Pro
#253I bet there’s gonna be a banger of a Mac Studio announced in June. Apple really stumbled into making the perfect hardware for home inference machines. Does any hardware company come close to Apple in terms of unified memory and single machines for high throughput inference workloads? Or even any DIY build? When it comes to the previous “pro workloads,” like video rendering or software compilation, you’ve always been…
Jeff Geerling doing that 1.5TB cluster using 4 Mac Studios was pretty much all the proof needed to demo how the Mac Pro is struggling to find any place any more. https://www.jeffgeerling.com/blog/2025/15-tb-vram-on-mac-stu...
That is the proof what is left is a workaround, just like pilling minis on racks because Apple left the server space.
Also why Swift nowadays has to have good Linux support, if app developers want to share code with the server.
Re: Apple discontinues the Mac Pro
#254I bet there’s gonna be a banger of a Mac Studio announced in June. Apple really stumbled into making the perfect hardware for home inference machines. Does any hardware company come close to Apple in terms of unified memory and single machines for high throughput inference workloads? Or even any DIY build? When it comes to the previous “pro workloads,” like video rendering or software compilation, you’ve always been…
As to better or cheaper homelab: depends on the build. AMD AI Max builds do exist, and they also use unified memory. I could argue the competition was, for a long time, selling much more affordable RAM, so you could get a better build outside Apple Silicon.
Re: Apple discontinues the Mac Pro
#255Earlier quoted context omitted.
Gaming isn't what people are using Mac Studios for. Thunderbolt also isn't a substitute for OCuLink.
Sure, but it’s probably reflective of the fact that GPUs generally aren’t PCIe-bandwidth bound. Also, TB5 and Oculink2 both use PCI 4.0 x4 links.
That's just blatantly wrong, the performance loss of GPUs is very well documented and gets worse as you go towards higher end models. We're talking 30%+ loss of performance here.
Re: Apple discontinues the Mac Pro
#256I bet there’s gonna be a banger of a Mac Studio announced in June. Apple really stumbled into making the perfect hardware for home inference machines. Does any hardware company come close to Apple in terms of unified memory and single machines for high throughput inference workloads? Or even any DIY build? When it comes to the previous “pro workloads,” like video rendering or software compilation, you’ve always been…
Agreed. I’m planning on selling my 512GB M3 Ultra Studio in the next week or so (I just wrenched my back so I’m on bed-rest for the next few days) with an eye to funding the M5 Ultra Studio when it’s announced at WWDC.
I can live without the RAM for a couple of months to get a good price for it, especially since Apple don’t sell that model (with the RAM) any more.
Re: Apple discontinues the Mac Pro
#257I bet there’s gonna be a banger of a Mac Studio announced in June. Apple really stumbled into making the perfect hardware for home inference machines. Does any hardware company come close to Apple in terms of unified memory and single machines for high throughput inference workloads? Or even any DIY build? When it comes to the previous “pro workloads,” like video rendering or software compilation, you’ve always been…
I don't think Apple just stumbled into it, and while I totally agree that Apple is killing it with their unified memory, I think we're going to see a pivot from NVidia and AMD. The biggest reason, I think, is: OpenAI has committed to enormous amount capex it simply cannot afford. It does not have the lead it once did, and most end-users simply do not care. There are no network effects. Anthropic at this point has com…
No one cares about Metal in that space, plus CUDA already has unified memory for a while.
https://docs.nvidia.com/cuda/cuda-programming-guide/04-speci...
Can we also stop giving Apple some prize for unified memory?
It was the way of doing graphics programming on home computers, consoles and arcades, before dedicated 3D cards became a thing on PC and UNIX workstations.
Re: Apple discontinues the Mac Pro
#258As someone who came from the SGI O2/Octane era when high-end workstations were compact, distinctive, and sexy, I’ve never really understood the allure of the Mac Pro, with the exception of the 2013 Mac Pro tube, which I owned (small footprint, quiet, and powerful). For me, aesthetics and size are important. That workstation on your desk should justify its presence, not just exist as some hulking box. When Apple relea…
As someone who worked on the M2 Mac Pro and has a real soft spot for it, I get it. It’s horrendously expensive and doesn’t offer much benefit over a Mac Studio and a thunderbolt pci chassis. My personal dream is that vms would support pci pass through and so you can just spin up a Linux vm and let it drive the gpus. But at that point, why are you buying a Mac? Opinions are my own obvs.
Under a comment regarding the O2/Octane (both of which I own :) era, I first read “vms” as VMS, not multiple instances of a VM…
Re: Apple discontinues the Mac Pro
#259Earlier quoted context omitted.
Just about everybody who isn't Nvidia dropped the ball, bigtime. Intel should have shipped their GPUs with much more VRAM from day one. If they had done this, they'd have carved out a massive niche and much more market share, and it would have been trivially simple to do. AMD should have improved their tools and software, etc. Apple should have done as you say. Google had nigh on a decade to boost TPU production, and…
> And thus Nvidia is, now quite durably, the most valuable company in the world. Nvidia is the most valuable company in the world right up until the AI bubble pops. Which, while it's hard to nail down when, is going to happen. I wouldn't call their position durable at all.
I might as well say that no, it is not going to happen.
As handwriting code is rapidly going out of fashion this year, it seems likely AI is coming for most of knowledge work next.
And who is to say that manual labor is safe for long?
Re: Apple discontinues the Mac Pro
#260Earlier quoted context omitted.
The top Mac Studio has six thunderbolt 5 ports, each of which is a PCIe 4.0 x4 link. Each is a 8GB/sec link in each direction, which is a lot. Going from x16 down to x4 has less than a 10% hit on games: https://www.reddit.com/r/buildapc/comments/sbegpb/gpu_in_pci...
Gaming isn't what people are using Mac Studios for. Thunderbolt also isn't a substitute for OCuLink.
Um, I have an M3 Ultra 512GB on my desk for development. Love me some Baldur’s Gate 3, everything turned up to 11…