There are so many used M4 Mac Minis for sale on marketplace for $600+ now that the people I guess got bored of the local models and decided to see if they could make their money back as they probably bought them when they were $399 on sale earlier this year. I lowball them every time.
Apple caught off guard by AI demand for Mac Mini and Mac Studio
81–90 of 636 posts
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#82I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…
IMO local models require a substantial amount of prompt+harness engineering to get in the neighborhood of what you'd get from a cloud model. Which isn't a bad thing, you'll learn a whole lot about how these things work. What you'll learn pretty quickly from said engineering is that there's a lot more to a good LLM than just the weights themselves. You need a good search provider (also self-hostable, but sounds easier…
What you’re describing is exactly what I’ve experienced in my time testing out this stuff locally, and I had a hard time figuring out what exactly to blame.
I keep hoping that one day some comment is going to paste a link to some kind of idiot-proof guide or piece of software that’s “90% as good as Claude but running local.”
And by 90% I don’t mean that the model is 90% as good or runs 90% as fast, more like all the other stuff you mentioned is set up out of the box.
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#83Earlier quoted context omitted.
> it seems really far off from the kind of experience even a basic $20/month subscription gets me. The $20/month subs are much stronger than the local models you can run, even with how far local models have advanced lately. The appeal of local models is that the data never leaves your network so you can feel safer putting sensitive content into it. It also feels “free” to use when you’ve already paid for the hardware…
It also takes some load off the AI data centers. IDK if that might be a concern for Apple or their AI partners.
It worsens the supply crunch, no? A unit you use sparingly vs that memory going into a GPU that serves many more people.
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#84I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…
In my experience so far, separately from privacy concerns there's a specific use case where cheap local shines--when unlimited shots on /goal with a dumber model is better than limited time with a smarter model. This looks less like the assisted-coding scenario that's commonly brought up as a good local scenario, because if you're searching for a fast and accurate solution to a single blocking problem, the bare minimum for a model that can do that is a 24GB dGPU or a 64GB Mac. Instead, this looks more like a Hermes agent on an Raspberry Pi driving OpenCode on an old gaming computer with just enough RAM and VRAM to handle an MoE, churning out something overnight that would quickly exhaust the subscription plans, like a knowledge graph for a large document corpus.
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#85Earlier quoted context omitted.
I don’t understand how that’s possible. They should have had a better idea of what was happening in the memory markets than pretty much any other entity.
Their universal RAM strategy is so obviously helpful for AI. (1) GPU/NPU CPU RAM copies eliminated. (2) All (most) RAM available for GPU/Neural, when local models are typically kneecapped by limited GPU RAM sizes vs. the much larger RAM options for M/Max/Pro/Ultras. They have been taking NPU's seriously on their phones, tablets and laptops since the M1. Then they enabled fully-connected RDMA for 4 x 512GB MacStudio's…
FWIW, the reported reason for OpenAI buying Macs has nothing to do with the memory by the sounds of it. Every single outlet I can find reporting on this seems to repeat that the intended use case is for agentic workloads and generating training data for reinforcement learning. They don't appear to be doing inference nor any sort of training AFAICT.
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#86Apple has a huge opportunity here to lead the market for machines to run local models if they step into it. Their stuff is already better than what nVidia is offering with stuff like the DGX Spark. It's a niche market but it's a market that overlaps heavily with professionals in the AI space and lead developers, so it's a market that gets them customers in those roles. If I were running Apple I'd call the RAM price b…
Apple doesn't design GPUs on-par with Nvidia's efficiency yet. They need an architectural overhaul to be a serious competitor, which is what I'm expecting is queued up for M7.
Nvidia has CUDA, AMD has CDNA, and Apple has... compute shaders, I guess?
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#87just a real bummer that they raised the prices so much
It definitely puts it out of the range of every day users but a non-insignificant proportion of people who use it for ai have become multi-millionaires because of ai. So there's definitely no shortage of people who have no problem paying those high prices.
> but a non-insignificant proportion of people who use it for ai have become multi-millionaires because of ai
I can't even name one person who fits this mold, let alone a non-insignificant proportion of people. Who are you thinking of?
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#88If I had to pick a product, I'd say an affordable 32GB mac would be the sweet spot for running local models that function well like Qwen 3.8. It's true, most people don't run models, but being the default platform for running open weights seems like it has plenty of advantages right now. Just like sales benefited from developers defaulting to MacOS for most open source languages like Ruby, Go, Rust, and TypeScript.
> If I had to pick a product, I'd say an affordable 32GB mac would be the sweet spot for running local models that function well like Qwen 3.8. 32GB is not enough RAM. I don't even own a device with less than 36GB at this point, and that device I only have because my employer is being cheap. 64GB is a reasonable starting point for running local LLMs + normal tasks. 128GB let's you really run most smaller models like…
You are mistaken. I'm running Qwen 3.7 28B 4bit (MLX) with a 200k context window and everything total is 32GB RSS.
Is this the best? No. That's why I said the sweet spot. Getting from 16GB macs to 32GB is perhaps possible. Jumping to 64GB or 128GB as the default is simply unreasonable right now.
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#89I imagine Apple could also grow their business in the EU by marketing to companies who want powerful AI features but can't leverage 3rd party AI services due to GDPR. If only they could come up with iOS, App Store, and developer policies that respect the sovereignty of the EU.
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#90Earlier quoted context omitted.
Their universal RAM strategy is so obviously helpful for AI. (1) GPU/NPU CPU RAM copies eliminated. (2) All (most) RAM available for GPU/Neural, when local models are typically kneecapped by limited GPU RAM sizes vs. the much larger RAM options for M/Max/Pro/Ultras. They have been taking NPU's seriously on their phones, tablets and laptops since the M1. Then they enabled fully-connected RDMA for 4 x 512GB MacStudio's…
FWIW, the reported reason for OpenAI buying Macs has nothing to do with the memory by the sounds of it. Every single outlet I can find reporting on this seems to repeat that the intended use case is for agentic workloads and generating training data for reinforcement learning. They don't appear to be doing inference nor any sort of training AFAICT.
For OpenAI, their data center archipelago is their own "local" and "personalized" AI.