Apple caught off guard by AI demand for Mac Mini and Mac Studio

Post by vablings » Mon, Aug 31, 2026, 7:50 PM UTC

ololobus wrote: post #186

Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

I was looking at $10k Mac Studio with M5 Ultra and 256 GB for local experiments, but then struggled to find what really good modern model I can fit into it. Yes, it can run a good dense 27B at Q8 with plenty of context, but what beyond that? IIUC, some Deepseek flash variants at Q4 are also feasible, but I am not sure if the quality will be good. They also don’t run that fast, like about 30 t/s So if I stay within 35…

Post by ryandrake » Mon, Aug 31, 2026, 7:52 PM UTC

nomel wrote: post #175

pizza234 wrote: post #127

Earlier quoted context omitted.

There are a few use cases that are (somewhat) surprisingly unsuited for cloud providers: - translations: cloud providers can bowdlerize (censor) bad words/content; also, if you want to do a translation for personal use of copyrighted materials, cloud providers may block it - image generation: generating drawings with a style that even just resembles a copyrighted one (ie. Disney) may be blocked by cloud providers - f…

Cloud does not mean censored. You can rent gpu time and run whatever model you want, with your data kept as private as any other cloud instance you personally run. Cloud is location, with (for some work) wayyyy cheaper access.

Post by reaperducer » Mon, Aug 31, 2026, 7:53 PM UTC

N_A_T_E wrote: post #22

dewey wrote: post #10

Earlier quoted context omitted.

> you know they have them internally... What makes you think that? There's a lot of data centers that sell you access to colocated Mac Mini's, they have added FileVault unlock via SSH in the boot process which also makes things easier. There's not that many reasons to run a Mac in the cloud unless you have some very specific Mac related workload.

At this point it’s a well known secret that Apple has real rack mount servers for their internal processes. They actually have officially released video of their servers in the WSJ report on their chip supply chain. https://forums.macrumors.com/threads/photos-of-apples-own-ne...

Post by bigyabai » Mon, Aug 31, 2026, 7:56 PM UTC

nicce wrote: post #123

bigyabai wrote: post #86

Earlier quoted context omitted.

Apple doesn't design GPUs on-par with Nvidia's efficiency yet. They need an architectural overhaul to be a serious competitor, which is what I'm expecting is queued up for M7. Nvidia has CUDA, AMD has CDNA, and Apple has... compute shaders, I guess?

> Apple doesn't design GPUs on-par with Nvidia's efficiency yet How much it matters in inference? Most GPUs have enough computing for that and the bottleneck is the RAM speed and size. And M5 Ultra is becoming to challenge this.

Post by redox99 » Mon, Aug 31, 2026, 7:59 PM UTC

Foobar8568 wrote: post #137

Xeoncross wrote: post #88

Earlier quoted context omitted.

You are mistaken. I'm running Qwen 3.7 28B 4bit (MLX) with a 200k context window and everything total is 32GB RSS. Is this the best? No. That's why I said the sweet spot. Getting from 16GB macs to 32GB is perhaps possible. Jumping to 64GB or 128GB as the default is simply unreasonable right now.

Memory used : 38GB, and I haven't even started a LLM nor podman, I always fight with memory when using LLM on my mac with 48gb. And I don't remember to have been able to have pushed to 200k context Qwen 3.6. 3.8 is running on my RTX 5090.