Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple caught off guard by AI demand for Mac Mini and Mac Studio

macrumors.com

Post reply on HN
Print view View original

291–300 of 636 posts

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#291

Post by wizee »

Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

For software development tasks, Qwen 3.8 27B is genuinely excellent, but you need 32+ GB of VRAM to run it well with decent context, and enough memory bandwidth and compute to run it at a decent pace. With an M5 Max Mac Studio, you can do that decently well.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#292

Post by etoxin »

saejox wrote: post #278

Most of those people will be dissapointed when they experience Q4 variants of those models getting stuck in loops. I would wait till the ram crisis is over to fetch a future 64gb ram gpu to run Q8 models. Cloud inference until than.

Switching from Q4 to Q8 was a game changer when I upgraded

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#293

Post by vardalab »

SamInTheShell wrote: post #161
Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

From what I’ve been seeing, the Mac studios do look like they have potential. I was looking to drop $10k-$15k on one until recently. After comparing a Radeon 7900 XTX vs Ryzen Halos 128GB vs M1 MacBook Pro 64Gb, I landed on just getting an external closure setup with Nvidia RTX 5090. The model I’m specifically targeting to use at high speeds is Qwen 3.8 27b @q4ks. This model actually proved to be good at coding (it s…

try ninfer once you get your 5090 https://github.com/Neroued/ninfer

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#294

Post by reilly3000 »

jasode wrote: post #68
Grombobulous wrote: post #19

I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…

>I realize I’m somewhat limited (16GB RTX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. I just ordered a new Mac Studio M5 Max 128GB $5899 ($6400 with tax) to be able to run the bigger "consumer size" models in the 70B parameter range (~96 GB). That said, I have no illusions that this expensive setup with a Qwen Flash coding LLM will be comparable t…

Keep the memory. You’ll be glad you did when you realize that you’re better off with a solid coding model plus a good voice model and also a lightweight all-rounder all running at once isn’t of loading dynamically (slowly). It also helps if you want to be able to run a browser, IDE, and container environment.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#295

Post by vardalab »

unsnap_biceps wrote: post #183
SamInTheShell wrote: post #161

Earlier quoted context omitted.

From what I’ve been seeing, the Mac studios do look like they have potential. I was looking to drop $10k-$15k on one until recently. After comparing a Radeon 7900 XTX vs Ryzen Halos 128GB vs M1 MacBook Pro 64Gb, I landed on just getting an external closure setup with Nvidia RTX 5090. The model I’m specifically targeting to use at high speeds is Qwen 3.8 27b @q4ks. This model actually proved to be good at coding (it s…

How are you actually using the local model? I've played with Qwen 3.8 27b on ollama and the coding harnesses (Claude Code and OpenCode) seem to fail way more often then using the cloud models. And by fail, I mean the edits don't apply cleanly, it goes to add python code, but doesn't indent it properly, or the edit doesn't apply and so it tries again and again and eventually wipes out a different function then it "int…

try using pi harness, hae not encountered these sort of problem myself also yuou can ask codex to look at the transcript and figure out the solutions to tool call failures that way

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#296

Post by dhosek »

esotericsean wrote: post #288

Imagine if Apple truly went the route of local AI and every Mac came with a fully local, open weight Siri. They could put Anthropic and OpenAI out of business.

Presumably this is on their roadmap. The Jobs-era thing would be to come in with something that people on HN would say, “yeah, I could do this myself with open source tools” but in practice falls into the category of “I could build this $30 thing for $15 for parts and then another $15 for the parts I had to replace because I screwed up the build the first time and then $30 to just buy the damn thing” that would really blow non-HN people away even if it is more expensive than most other local computing systems, but I don’t know what the Ternus-era Apple will be like.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#297

Post by SR2Z »

NetMageSCW wrote: post #247
SR2Z wrote: post #164

Earlier quoted context omitted.

This kind of stuff is available off the shelf at any major cloud provider.

Guaranteed privacy is available off the shelf?

Yes, insofar as any "guaranteed" is cloud providers just saying "we pinky swear to not decrypt your resources!"

I know what I'm talking about here - trust me when I say that the difference between a "government" or "large corpo" account and "some dude" is literally a few flags to prevent resources being created in the wrong regions. It's nothing that a random customer can't do themselves.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#298

Post by Danox »

What’s going on isn’t Apple behind in AI model building I thought I read that somewhere on the MacRumors site in the last two years, that Apple is behind its tech peers and Apple might as well just close the doors.

I always thought Apple was in a good position because unlike their peers they didn’t burn billions of dollars trying to build an AI model that has no financial moat around it.

I still think they’re in a good position in comparison to their tech peers and we will know even more when some of the new computers get into the hands of some of the tech reviewers.

I believe the new computer’s will be pretty good hardware wise what I’m interested in, is the Apple software support for connecting several Mac computers together, and some of the other (new?) software that Apple may have written in house to support those who want to run AI software locally that is just as important as the new hardware.

Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio

#299

Post by mcotton »

ern_ave wrote: post #185
mcotton wrote: post #117

Earlier quoted context omitted.

A simple example. I have an older M2 Mac mini that does the OCR and visual description of all my screenshots. Screenshots are stored on my NAS. I like to screenshot things as a quick way to remember. They are things that I would not be comfortable sending a cloud provider (customer data, prototype screenshots, bank dispute details). It runs Qwen3.5:9b and glm5.2-ocr with Ollama and uses about 10GB of RAM. It automati…

Would you mind writing that up in more detail and posting it somewhere? It sounds pretty interesting.

I will write about it at some point but can't get to it today.

You can grab the code here: https://github.com/mcotton/listing

I run it in Docker on my laptop or Synology NAS. It uses the OpenAI/Ollama URL schema for processing.

Post reply on HN