Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple discontinues the Mac Pro

9to5mac.com

Post reply on HN
Print view View original

611–620 of 672 posts

Re: Apple discontinues the Mac Pro

#611

Post by NetMageSCW »

wpm wrote: post #192
dangus wrote: post #14

> Serviceable, repairable, upgradable Macs are officially a thing of the past. Well, not exactly. Apple’s desktop Macs actually all have modular SSD storage, and third parties sell upgrade kits. And it’s not like Thunderbolt is a slouch as far as expandability. I can see why the Mac Pro is gone. Yeah, it has PCIe slots…that I don’t really think anyone is using. It’s not like you can drop an RTX 5090 in there. The lat…

Thunderbolt absolutely is a slouch. Every PCIe card I have requires it's own $150+ PCIe to Thunderbolt Dock and its own picoPSU plus 12V power supply. External PCIe is convenient for portables. Not for desktops. It's a piss-poor replacement for a proper PCIe slot.

Why don’t you just get a multi-slot PCIe box?

Re: Apple discontinues the Mac Pro

#612

Post by AnthonyMouse »

actionfromafar wrote: post #309
AnthonyMouse wrote: post #263

Earlier quoted context omitted.

> A huge reason for this is Apple needs unified memory to keep their money maker (laptops) profitable and performant None of the things people care about really get much out of "unified memory". GPUs need a lot of memory bandwidth, but CPUs generally don't and it's rare to find something which is memory bandwidth bound on a CPU that doesn't run better on a GPU to begin with. Not having to copy data between the CPU an…

> Not having to copy data between the CPU and GPU is nice on paper but again there isn't much in the way of workloads where that was a significant bottleneck. Isn't that also because that's world we have optimized workloads for? If the common hardware had unified memory, software would have exploited that I imagine. Hardware and software is in a co-evolutionary loop.

Sort of?

Part of the problem is that there is actually a reason for the distinction, because GPUs need faster memory but faster memory is more expensive, so then it makes sense to have e.g. 8GB of GDDR for the GPU and 32GB of DDR for the CPU, because that costs way less than 40GB of GDDR. So there is an incentive for many systems to exist that do it that way, and therefore a disincentive to write anything that assumes copying between them is free because it would run like trash on too large a proportion of systems even if some large plurality of them had unified memory.

A sensible way of doing this is to use a cache hierarchy. You put e.g. 8GB of expensive GDDR/HBM on the APU package (which can still be upgraded by replacing the APU) and then 32GB of less expensive DDR in slots on the system board. Then you have "unified memory" without needing to buy 40GB of GDDR. The first 8GB is faster and the CPU and GPU both have access to both. It's kind of surprising that this configuration isn't more common. Probably the main thing you'd need is for the APU to have a direct power connector like a GPU so you're not trying to deliver most of a kilowatt through the socket in high end configurations, but that doesn't explain why e.g. there is no 65W CPU + 100W GPU with a bit of GDDR to be put in the existing 170W AM5 socket.

However, even if that was everywhere, it's still doesn't necessarily imply there are a lot of things that could do much with it. You would need something that simultaneously requires more single-thread performance than you can get from a GPU, more parallel computation than you can get from a high-end CPU, and requires a large amount of data to be repeatedly shared between those subsets of the computation. Such things probably exist but it's not obvious that they're very common.

Re: Apple discontinues the Mac Pro

#613

Post by brailsafe »

ua709 wrote: post #592
brailsafe wrote: post #554

Earlier quoted context omitted.

I'd argue that some of those are more consumption and activity than hobby depending on how they're engaged with, and that people use the word "hobby" too loosely, but would agree that Americans in-particular consume at obscene rates. Golf equipment, mountaineering equipment, skiing and snowboarding lift tickets and gear, a single excessive graphics card that's only used for increasing frame rates marginally, or basic…

Those are the prices for just buying equipment, which at least retain some kind of value. 3 million+ American kids are enrolled in competitive soccer with annual clubs dues between $1K and $5K, and that money is just gone at the end of the year. Basically none of those kids are going to have a career in soccer, so it's clearly a hobby, and everyone knows it. And soccer isn't even the most popular sport!

Ya, I guess that's another category entirely. The cost of enrolling a kid in anything, potential travel involved etc..

Re: Apple discontinues the Mac Pro

#614

Post by NetMageSCW »

jeffbee wrote: post #159

I guess A/V pros are used to getting screwed constantly, but it must be really irritating to face the prospect of eventually having to move PCI add-in cards to TB5 enclosures that cost $1000 per slot.

I see retail 3 slots for $1800, so a lot cheaper than you think. They can move to a Studio and buy a box for less than a Mac Pro replacement would cost.

Re: Apple discontinues the Mac Pro

#615

Post by JumpCrisscross »

jiggawatts wrote: post #219
JumpCrisscross wrote: post #115

Earlier quoted context omitted.

> Apple is counting on something else: model shrink The most powerful AI interactions I've had involved giving a model a task and then fucking off. At that point, I don't actually care if it takes 5 minutes or an hour. I've cued up a list of background tasks it can work on, and that I can circle back to when I have time. In that context, smaller isn't even the virtue at hand–user patience is. Having a machine that wo…

What "tooling" do you use to let AIs work unattended for long periods?

> What "tooling" do you use to let AIs work unattended for long periods?

Claude and Kagi Assistant. I tried tooling up a multi-model environment in Ollama and it was annoying. It's just searching the web, building models and then running a test suite against the model to refine it.

Re: Apple discontinues the Mac Pro

#616

Post by angoragoats »

lowbloodsugar wrote: post #507
angoragoats wrote: post #479

Earlier quoted context omitted.

> The mac will just work for models as large as 100B, can go higher with quantized models. And power draw will be 1/5th as much as the 3090 setup. This setup will work for 100B models as well. And yes, the Mac will draw less power, but the Nvidia machine will be many times faster. So depending on your specific Mac and your specific Nvidia setup, the performance per watt will be in the same ballpark. And higher absolu…

How much does it cost to have an electrician wire up 240v circuit just to power the thing?

The machine I’m describing works just fine on a dedicated 15A 120V circuit.

Re: Apple discontinues the Mac Pro

#617

Post by angoragoats »

rybosworld wrote: post #510
angoragoats wrote: post #479

Earlier quoted context omitted.

> The mac will just work for models as large as 100B, can go higher with quantized models. And power draw will be 1/5th as much as the 3090 setup. This setup will work for 100B models as well. And yes, the Mac will draw less power, but the Nvidia machine will be many times faster. So depending on your specific Mac and your specific Nvidia setup, the performance per watt will be in the same ballpark. And higher absolu…

I think it's bad form to say "citation needed" when your original claim didn't include citations. Regardless - there's a difference between training and inference. And pytorch doesn't magically make 5 gpus behave like 1 gpu.

> I think it's bad form to say "citation needed" when your original claim didn't include citations.

I apologize, but using multiple GPUs for inference (without any sort of “daisy chaining”) is something that’s been supported in most LLM tooling for a long time.

> Regardless - there's a difference between training and inference.

No one brought up training vs. inference to my knowledge, besides you — I was assuming the machine was for inference, because my experience building a machine like the one I described was in order to do inference. If you want to train models, I know less about that, but I’m pretty sure the tooling does easily support multiple GPUs.

> And pytorch doesn't magically make 5 gpus behave like 1 gpu.

I never said it was magic, I just said it was supported, which it is.

Re: Apple discontinues the Mac Pro

#618

Post by angoragoats »

whywhywhywhy wrote: post #567
rybosworld wrote: post #461

Earlier quoted context omitted.

I don't see how 5x 3090's is a better option than an M3 Ultra Mac studio. The mac will just work for models as large as 100B, can go higher with quantized models. And power draw will be 1/5th as much as the 3090 setup. You can certainly daisy chain several 3090's together but it doesn't work seamlessly.

> You can certainly daisy chain several 3090's together It's not "daisy chaining" 3090 has NVLink.

FWIW I have never used NVLink, and I’m not sure why people are bringing up “daisy chaining” because as far as I’m aware that is not a thing with modern GPUs at all.

Re: Apple discontinues the Mac Pro

#619

Post by bigyabai »

mjburgess wrote: post #599
heraldgeezer wrote: post #552

Earlier quoted context omitted.

You can keep CPU and RAM for way longer than the GPU if you game... Your point kind of disproves your point.

https://store.steampowered.com/hwsurvey/videocard/

Case in point. Most popular cards are discrete GPUs that can be upgraded.

Re: Apple discontinues the Mac Pro

#620

Post by plagiarist »

embedding-shape wrote: post #402
plagiarist wrote: post #392

Earlier quoted context omitted.

Not quite, what is the vRAM bandwidth of each? The bandwidth is a huge contributor to LLM performance.

AFAIK, for the unified bandwidth, it depends mostly on the CPU, for M4 Max (I think it's the default today?) it does ~550 GB/s, while GB10 does ~270 GB/s, so about a 2x difference between the two. For comparison, RTX Pro 6000 does 1.8 TB/s, pretty much the same as what a 5090 does, which is probably the fastest/best GPUs a prosumer reasonable could get.

Granted, it won't be competitive against the flagship dGPUs. Nevertheless, that ~2x is a pretty huge difference in similarly priced offerings.
Post reply on HN