Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple discontinues the Mac Pro

9to5mac.com

Post reply on HN
Print view View original

471–480 of 672 posts

Re: Apple discontinues the Mac Pro

#471

Post by dabockster »

traceroute66 wrote: post #437
jiwidi wrote: post #426

Earlier quoted context omitted.

tell me what pc with an nvidia gpu can you buy with same memory and performance. I never liked apple hardware, but they are now untouchable since their shift to own sillicon for home hardware.

> tell me what pc with an nvidia gpu can you buy with same memory and performance. And power consumption ! The performance per watt of Apple is unmatched.

This needs to be sold as the big ticket item for low level devs. Their chips are some of the most power efficient chips on the market right now.

Hoping they release a blade server version somehow.

Re: Apple discontinues the Mac Pro

#472

Post by kube-system »

bigyabai wrote: post #201
kube-system wrote: post #101

Earlier quoted context omitted.

It's not like Apple soldered some plain old DDR5 to a PCB to be difficult: 1. It's TSMC's InFO_POP, which has significant performance benefits. 2. There weren't even any modules that existed for LPDDR until very recently. (and while the A18 was being designed, it didn't exist) 3. The power/price/performance/thermals they are able to achieve with this configuration is not possible with socketed RAM. You are asking the…

The "Mac memory is special" shtick is getting old. macOS is a heavy OS, 8gb is an objective liability to the SSD lifespan.

Not a single thing in my comment has anything to do with MacOS.

Re: Apple discontinues the Mac Pro

#473

Post by dragonwriter »

NetMageSCW wrote: post #450
dragonwriter wrote: post #313

Earlier quoted context omitted.

> Apple really stumbled into making the perfect hardware for home inference machines For LLMs. For inference with other kinds of models where the amount of compute needed relative to the amount of data transfer needed is higher, Apple is less ideal and systems worh lower memory bandwidth but more FLOPS shine. And if things like Google’s TurboQuant work out for efficient kv-cache quantization, Apple could lose a lot o…

Or just mean that you could run a 5x bigger model on Apple than before.

Well, since its kv-cache that TurboQuant optimizes, it means five times bigger context fits into RAM, all other things being equal, not a five times bigger model. But, sure, with any given context size and the same RAM available, you can instead fit a bigger model—which also takes more compute to get the same performance.

Anything that increases the necessary compute to fully utilize RAM bandwidth in optimal LLM serving weakens Apples advantage for that.

Re: Apple discontinues the Mac Pro

#474

Post by dabockster »

rafram wrote: post #316
whywhywhywhy wrote: post #303

Earlier quoted context omitted.

> But inference is unique because its performance scales with high memory throughput, and you can’t assemble that by wiring together off the shelf parts in a consumer form factor. Nvidia outperforms Mac significantly on diffusion inference and many other forms. It’s not as simple as the current Mac chips are entirely better for this.

But where are you going to find an Nvidia GPU with 128+ GB of memory at an enthusiast-compatible price?

You don’t need it if you use llamacpp on Windows, or if you compile it on Linux with CUDA 13 and the correct kernel HMM support, and you’re only using MoE models (which, tbh, you should be doing anyways).

Re: Apple discontinues the Mac Pro

#475

Post by BeetleB »

A_D_E_P_T wrote: post #61
readitalready wrote: post #59

Apple really dropped the ball here. They had every ability to make something competitive with Nvidia for AI training as well as inference, by selling high end multi GPU Mac Pro workstations as well as servers, but for some reason chose not to. They had the infrastructure and custom SoCs and everything. What a waste. It really could have been a bigger market for them than even the iPhone.

Just about everybody who isn't Nvidia dropped the ball, bigtime. Intel should have shipped their GPUs with much more VRAM from day one. If they had done this, they'd have carved out a massive niche and much more market share, and it would have been trivially simple to do. AMD should have improved their tools and software, etc. Apple should have done as you say. Google had nigh on a decade to boost TPU production, and…

Trust me: If Intel could, it would.

From inside news: They were not breaking even on their existing GPUs. The strategy was to take a loss just to have a presence in the space.

Re: Apple discontinues the Mac Pro

#476

Post by dabockster »

audunw wrote: post #390
andrewl-hn wrote: post #332

This would probably push some high-end audio professionals away from Logic. One of the niches Mac Pro has been popular is audio production. And with cheesegrader the ability to slot in many-many different audio interfaces into a box instead of dangling out to various PCIe enclosures has been a big win. Here's a good video how it looks like: https://www.youtube.com/watch?v=kIQINCWMd6I&list=PLi2i2YhL6o... (at 1:40 Neil…

Feels like it'd just create a market for a big rack-mountable multi-bay PCIe enclosure, with its own internal power supply, that you could connect with one ore more thunderbolt cable. I don't see any reason why a solution built around a Mac Studio should have to be significantly more cluttered. I don't know if such a solution exists right now, but I'm thinking there's a fair chance it will soon as the Mac Pro disappe…

Thunderbolt is really an unsung hero here. It is surprisingly nice to be able to move various components around my desk that would have otherwise sat in a huge tower hogging all the PCIe slots they can find.

Re: Apple discontinues the Mac Pro

#477

Post by dabockster »

BeetleB wrote: post #475
A_D_E_P_T wrote: post #61

Earlier quoted context omitted.

Just about everybody who isn't Nvidia dropped the ball, bigtime. Intel should have shipped their GPUs with much more VRAM from day one. If they had done this, they'd have carved out a massive niche and much more market share, and it would have been trivially simple to do. AMD should have improved their tools and software, etc. Apple should have done as you say. Google had nigh on a decade to boost TPU production, and…

Trust me: If Intel could, it would. From inside news: They were not breaking even on their existing GPUs. The strategy was to take a loss just to have a presence in the space.

Intel could position their cards as strong for certain workloads. They had AV1 support first in market, for example.

Re: Apple discontinues the Mac Pro

#478

Post by deaddodo »

Detrytus wrote: post #451
angoragoats wrote: post #443

Earlier quoted context omitted.

This has changed since Sam Altman started buying up all the chip supply, raising prices on memory, storage, and GPUs for everyone, but it used to be the case that you could build a PC that was both cheaper and faster than a Mac for LLM inference, with roughly equal performance per watt. You would use multiple *90-series GPUs, throttled down in terms of power. Depending on the GPU, the sweet spot is between 225-350W,…

With 6-7 GPUs and EPYC cpu it will also cost 2-3x more than a Mac Studio.

I think OP’s point was that it would do more than 2-3x the workload, thus them stating “blow it out of the water” and specifying “performance-per-watt”.

Re: Apple discontinues the Mac Pro

#479

Post by angoragoats »

rybosworld wrote: post #461
angoragoats wrote: post #445

Earlier quoted context omitted.

You can still buy used 3090 cards on ebay. 5 of them will give you 120GB of memory and will blow away any mac in terms of performance on LLM workloads. They have gone up in price lately and are now about $1100 each, but at one point they were $700-800 each.

I don't see how 5x 3090's is a better option than an M3 Ultra Mac studio. The mac will just work for models as large as 100B, can go higher with quantized models. And power draw will be 1/5th as much as the 3090 setup. You can certainly daisy chain several 3090's together but it doesn't work seamlessly.

> The mac will just work for models as large as 100B, can go higher with quantized models. And power draw will be 1/5th as much as the 3090 setup.

This setup will work for 100B models as well. And yes, the Mac will draw less power, but the Nvidia machine will be many times faster. So depending on your specific Mac and your specific Nvidia setup, the performance per watt will be in the same ballpark. And higher absolute performance is certainly a nice perk.

> You can certainly daisy chain several 3090's together but it doesn't work seamlessly.

Citation needed; there's no "daisy chaining" in the setup I describe, and low level libraries like pytorch as well as higher level tools like Ollama all seamlessly support multiple GPUs.

Re: Apple discontinues the Mac Pro

#480

Post by zadikian »

chatmasta wrote: post #165

I bet there’s gonna be a banger of a Mac Studio announced in June. Apple really stumbled into making the perfect hardware for home inference machines. Does any hardware company come close to Apple in terms of unified memory and single machines for high throughput inference workloads? Or even any DIY build? When it comes to the previous “pro workloads,” like video rendering or software compilation, you’ve always been…

What part of your workflow relies on home LLM inference?
Post reply on HN