Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple caught off guard by AI demand for Mac Mini and Mac Studio

macrumors.com

Post reply on HN
Print view View original

121–130 of 630 posts

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#121

Post by e28eta »

Xeoncross wrote: post #88
tristor wrote: post #50

Earlier quoted context omitted.

> If I had to pick a product, I'd say an affordable 32GB mac would be the sweet spot for running local models that function well like Qwen 3.8. 32GB is not enough RAM. I don't even own a device with less than 36GB at this point, and that device I only have because my employer is being cheap. 64GB is a reasonable starting point for running local LLMs + normal tasks. 128GB let's you really run most smaller models like…

You are mistaken. I'm running Qwen 3.7 28B 4bit (MLX) with a 200k context window and everything total is 32GB RSS. Is this the best? No. That's why I said the sweet spot. Getting from 16GB macs to 32GB is perhaps possible. Jumping to 64GB or 128GB as the default is simply unreasonable right now.

I have a similar machine, and briefly poked at running a local LLM, but got discouraged after a couple days. The quality, responsiveness, and impact on the rest of the system didn’t seem worth it to me.

What sorts of things are you doing with the local LLM? Anything interactive? Should I take another look?

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#122

Post by caycep »

yardie wrote: post #70
setgree wrote: post #14

It's fun to see that even an extremely large company can find unexpected product market fit [0]. Per this article, "The company reportedly did not possess an engineering team dedicated to business customers or staff focused on developer relations, and lacked an enterprise AI strategy." That sounds insane in retrospect, but I think there's just inherent uncertainty in what people actually need and will use things for.…

You should listen to the podcast Acquired, specifically Nvidia and then Jensen Huang. They basically lucked into AI. Some researcher was using Nvidia gaming cards, and reached out to them about questions on CUDA. That email eventually turned them into a trillion dollar question.

to their credit, there was a lot of work behind "luck". Jensen showed up in person in 2017 in NEURIPS and he and likely a lot of his top brass basically sat down and read the entire conference proceedings/abstracts; there was likely a lot of work behind the scenes to behind the ML research pivot.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#123

Post by nicce »

bigyabai wrote: post #86
api wrote: post #26

Apple has a huge opportunity here to lead the market for machines to run local models if they step into it. Their stuff is already better than what nVidia is offering with stuff like the DGX Spark. It's a niche market but it's a market that overlaps heavily with professionals in the AI space and lead developers, so it's a market that gets them customers in those roles. If I were running Apple I'd call the RAM price b…

Apple doesn't design GPUs on-par with Nvidia's efficiency yet. They need an architectural overhaul to be a serious competitor, which is what I'm expecting is queued up for M7. Nvidia has CUDA, AMD has CDNA, and Apple has... compute shaders, I guess?

> Apple doesn't design GPUs on-par with Nvidia's efficiency yet

How much it matters in inference? Most GPUs have enough computing for that and the bottleneck is the RAM speed and size. And M5 Ultra is becoming to challenge this.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#124

Post by notnullorvoid »

colingauvin wrote: post #114
hhh wrote: post #104

Earlier quoted context omitted.

this is insanely misleading, you can't run anything close to current chatgpt or gemini on local hardware

GLM 5.3 Flash? Qwen 3.8 Flash Next? I believe those both are as good as the best Gemini, competitive with Terra.

Yes they are quite good, but are not able to run on a 16GB RX 9070.

Quantized Qwen 3.8 Flash Next could maybe run eventually on that card with a highly optimized inference engine that dynamically caches the hottest layer experts. Even then you run into some hard limits.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#125

Post by procgen »

AdmiralAsshat wrote: post #46

Mac Mini's were really nice HTPC candidates, too, before the AI boom. Like all things genuinely useful and affordable, they were snatched from the hands of normal consumers by a bunch of schmucks chasing the latest gold rush.

Our Blessed Homeland / Their Barbarous Wastes

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#126

Post by adastra22 »

cyclopeanutopia wrote: post #96
rc1 wrote: post #35

Earlier quoted context omitted.

The article implies the demand is for running locally. I’m not convinced, at least with a mac minis. Most people I know and myself buy the mini as it is always on, easy to setup, and isolated from my main computer which is a laptop. The mini is driving the use of the $20/month subscriptions.

Why not use rpi then?

In my case, I dev on macOS. The env the agent runs in is the same as my dev laptop, configured and in sync. Has access to all the same tools and environment as I would on my laptop.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#127

Post by pizza234 »

Aurornis wrote: post #25
Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

> it seems really far off from the kind of experience even a basic $20/month subscription gets me. The $20/month subs are much stronger than the local models you can run, even with how far local models have advanced lately. The appeal of local models is that the data never leaves your network so you can feel safer putting sensitive content into it. It also feels “free” to use when you’ve already paid for the hardware…

There are a few use cases that are (somewhat) surprisingly unsuited for cloud providers:

- translations: cloud providers can bowdlerize (censor) bad words/content; also, if you want to do a translation for personal use of copyrighted materials, cloud providers may block it

- image generation: generating drawings with a style that even just resembles a copyrighted one (ie. Disney) may be blocked by cloud providers - for example, generating old cartoons style with GPT may not be possible.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#128

Post by DaSHacka »

iLoveOncall wrote: post #95
Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

I might be wrong but subscription models don't give you API access. I'm only interested in API access when it comes to personal use, so local models running for free makes sense for me. That said I have an RTX 5090, not a Mac Mini, so it's not exactly the same level of performance... The latest open models run at 200 tpm at around 30B params.

Only Anthropic does that AFAIK, at least I can use my $20/month Codex and Kimi subscriptions in pi.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#129

Post by BatFastard »

gcoakes wrote: post #120
Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

I have a RX 9070 also. I run llama-swap with a fill-in-middle 7B model, local 9B model, and it proxies up to OpenRouter for the bigger stuff. I think that's where the sweet spot is right now. I've spent $2 in the last 2 weeks on OpenRouter. I've been trying to only use the medium sized models that I would otherwise be able to run on a nice local setup. That nice local setup would cost ~$4k. I don't know what the oper…

I made the same choice, aside from privacy concerns, you can not locally host a cost competitive model.

But I am looking forward to lower hardware costs!

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#130

Post by TechSquidTV »

Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

In my limited experience, not quite yet but we are damn close. Qwen 3.8 27b is it. If I could run this as a decent speed, I would no longer need cloud models at all. I'm actually currently trying it out in the cloud to pay for the inference speed but the model is fully runnable at home.

I realistically costs $5-10k to replicate a ChatGPT like agent. And it doesn't scale.

That's still really close. And models and quantization etc keep improving.

I'm absolutely positive that I'll be switching to mostly local AI in the next 5 years.

Post reply on HN