Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple caught off guard by AI demand for Mac Mini and Mac Studio

macrumors.com

Post reply on HN
Print view View original

71–80 of 636 posts

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#71

Post by D13Fd »

Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

I’ve been running DeepSeek 4 Flash, Qwen 27B and Qwen 9B on local hardware. They work well for coding and document review tasks. I think Qwen 9B local on a 5090 might be legitimately helpful for small task agents in omp, since it is ridiculously fast. But my motivation is that I have data that I unfortunately can’t share with 3rd parties.

I have been eyeing a 512 GB Mac 5 Ultra to run full DS4 pro locally, which I expect would be pretty amazing as far as quality/recall. The only downside is that the speed is a lot slower than something like 27B on the 5090.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#72

Post by pizzafeelsright »

Nevermark wrote: post #65
adjejmxbdjdn wrote: post #29

Earlier quoted context omitted.

I don’t understand how that’s possible. They should have had a better idea of what was happening in the memory markets than pretty much any other entity.

Their universal RAM strategy is so obviously helpful for AI. (1) GPU/NPU CPU RAM copies eliminated. (2) All (most) RAM available for GPU/Neural, when local models are typically kneecapped by limited GPU RAM sizes vs. the much larger RAM options for M/Max/Pro/Ultras. They have been taking NPU's seriously on their phones, tablets and laptops since the M1. Then they enabled fully-connected RDMA for 4 x 512GB MacStudio's…

I am curious about the corporate disconnect from the frontline to the generals.

While the company I am in is embracing AI the disconnect and delay between what is available and possible versus what is approved and permitted is a three month window. The State employees I speak to are just now getting around to writing their usage policies for internal AI usage.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#74

Post by martythemaniak »

This may sound a little wacky, but one potential use case I'm considering is robotics. Say you want to use a fine-tuned mid-size VLM model right on the robot. You pretty much have to use the Jetson line, which has a great ecosystem and everything (depth cameras, lidar, SLAM, small segmentation models, etc) runs on it, but it's gets very expensive very fast if you want to run LLMs on it. The Jetson Thor lines are 3-5k depending on memory etc.

One very efficient option today is to have the cheapest Jetson (Orin Nano) run the classical robotics stack, then have a base mac mini run nothing but the VLM. The Mac mini is considerably cheaper and faster at these workloads than the mid-range Jetsons.

I think this wonky situation is because Apple us under immense consumer pressure to absorb the ridiculous memory prices, while the Jetson is aimed at "business" and much more likely to fluctuate with the market. Last year I bought a Jetson Orin Nano 8GB for $375CAD, today that official nVidia Amazon page is out of stock and other sellers have it listed for $900-$1100CAD. Absolutely bonkers pricing.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#75

Post by catoc »

paxys wrote: post #11

I really hope with Ternus taking the helm Apple starts to remember that it has products outside of iPhone.

I wish they would stop with the new-iPhone-every-year nonsense and refocus on quality, fix some bugs… but yeah, not gonna happen

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#76

Post by articulatepang »

BurningFrog wrote: post #32
Aurornis wrote: post #25

Earlier quoted context omitted.

> it seems really far off from the kind of experience even a basic $20/month subscription gets me. The $20/month subs are much stronger than the local models you can run, even with how far local models have advanced lately. The appeal of local models is that the data never leaves your network so you can feel safer putting sensitive content into it. It also feels “free” to use when you’ve already paid for the hardware…

It also takes some load off the AI data centers. IDK if that might be a concern for Apple or their AI partners.

Surely this is not something that motivates the vast majority of people using local LLMs.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#77

Post by articulatepang »

SaltyBackendGuy wrote: post #23
setgree wrote: post #14

It's fun to see that even an extremely large company can find unexpected product market fit [0]. Per this article, "The company reportedly did not possess an engineering team dedicated to business customers or staff focused on developer relations, and lacked an enterprise AI strategy." That sounds insane in retrospect, but I think there's just inherent uncertainty in what people actually need and will use things for.…

Maybe a bit of hindsight bias / the outside view here, but I feel like they're completely asleep if they didn't anticipate strong demand for this specific use case.

I think a reasonable story could have been told that goes like this: local models aren’t as good as frontier models with a $20/month subscription, and the hardware costs a lot. So only a few enthusiasts will buy Apple machines for this purpose.

This story turned out to be false but I think smart, reasonable people a couple years ago could have believed it with conviction. It doesn’t really seem like “completely asleep” to me.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#78

Post by ivewonyoung »

Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

> 16GB RTX 9070

What's an RTX 9070? Do you mean the RX 9070 or RTX 5070?

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#79

Post by Grombobulous »

ivewonyoung wrote: post #78
Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

> 16GB RTX 9070 What's an RTX 9070? Do you mean the RX 9070 or RTX 5070?

Oops yeah I meant RX 9070, fixed it!

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#80

Post by Grombobulous »

fisle wrote: post #34
Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

Could you elaborate on your experience with local models on your card? I've been thinking of upgrading to 9070 XT, and was thinking the 16GB would be okay-ish to at least run something usable locally, no?

I’m the parent of this thread, the person with the with the RX 9070.

My understanding would be that if you’re interested in this sort of card for AI that you should go with the AI PRO R9700, which is basically the professional version of the RX 9070XT but with 32GB of memory.

It’s significantly more money but not crazy like a 5090.

I just happen to have the 9070XT primarily for gaming purposes.

I’m not quite sure how to describe my experience using it other than “rudimentary,” and a lot of that is on me for not really understanding the best way to set it up.

Post reply on HN