Earlier quoted context omitted.
Why not use rpi then?
In my case, I dev on macOS. The env the agent runs in is the same as my dev laptop, configured and in sync. Has access to all the same tools and environment as I would on my laptop.
[dead]
Apple caught off guard by AI demand for Mac Mini and Mac Studio
311–320 of 636 posts
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#312Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#313I’m curious to know if these local AI setups are legitimately useful compared to cloud. I’ve struggled a lot to get something useful out of the hardware I have. I realize I’m somewhat limited (16GB RX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. Any tips anyone might have are appreciated! I’d love to be local first and would be willing to buy hardw…
Local inference can't compete with cloud on speed, intelligence and economics. It's all about freedom, privacy, control, sovereignty.
It's about not having to accept any of the stupid "terms" of the corporations. It's about doing things the big labs don't allow you to do, like cybersecurity stuff, or even just chatting with the AI about some wrongthink.
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#314Earlier quoted context omitted.
You should listen to the podcast Acquired, specifically Nvidia and then Jensen Huang. They basically lucked into AI. Some researcher was using Nvidia gaming cards, and reached out to them about questions on CUDA. That email eventually turned them into a trillion dollar question.
What year are you talking about? When I was in grad school, around 2007, Nvidia was aggressively marketing GPUs for high performance computing. They would go to campuses, talk to professors, etc. Yes, the whole Deep Learning thing was luck, but as with most lucky things, they ensured they were positioned to capitalize on it.
Probably cerca 2014 as that's when AlexNet was released, demonstrating that neural networks could beat traditional ML models at image recognition tasks. I recall the researchers used Cuda to optimize their training setup.
AlexNet kicked off a new wave of research around neural networks by demonstrating they could be scaled well and trained on GPUs.
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#315Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#316There is a lot of "AI demand" that isn't just running inference on an LLM whose weights you downloaded. I'm training a model using reinforcement learning with self-play. I can and do use vast.ai when scaling but for experiments it's far faster, and cheaper, to run it locally until the bugs are all figured out. Just provisioning a new instance and copying the relevant checkpoints and things can take 25 minutes. It's z…
A C-level executive I know is getting a top-of-the-line new Mac simply to function as a personal build server and host for agentic coding instances - they are able to orchestrate so many parallel projects that they're hitting RAM limits from sessions and the builds and local test runs they're kicking off (largly unsupervised). Before AI, they'd only had a MacBook Air; this completely changes their workflows. They talk about how many other executives they've met are equally giddy at having gone from coding few to no projects themselves, to coding more projects in parallel than any of their respective pre-AI technical colleagues.
I'd suspect that agentic coding has birthed so many new effective engineers, that the entire dynamics of demand for high-end machines have been upended.
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#317Earlier quoted context omitted.
The options available across the board are getting cheaper and better all the time. There is no reason to believe that equivalent level model output will be more expensive in 12 months, let alone almost 4 years from now. Of all the good reasons to use local AI (privacy, etc), worrying about not having access to cheap models in 4 years is not one of them.
The big providers are losing on average tens of billions a year on these services, so yes prices must go up. Even Moore’s won’t help in the medium-term due to shortages and difficulty/reluctance to vastly increase capacity.
You can buy tokens on OpenRouter right now from companies who serve tokens as a business and who are not selling at a loss.
The frontier labs have very high prices for inference. The prices are actually going down, not up.
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#318Earlier quoted context omitted.
>I realize I’m somewhat limited (16GB RTX 9070), but still, it seems really far off from the kind of experience even a basic $20/month subscription gets me. I just ordered a new Mac Studio M5 Max 128GB $5899 ($6400 with tax) to be able to run the bigger "consumer size" models in the 70B parameter range (~96 GB). That said, I have no illusions that this expensive setup with a Qwen Flash coding LLM will be comparable t…
Serious question: why not run DGX Spark or Framework Desktop, at 30%-50% lower cost?
[dead]
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#319Earlier quoted context omitted.
A new base model mac mini is $900. That is 45 month of Gemini. Gemini 4.7 Flash will give better OCR results that Qwen or GLM w/ 10GB.
This is such a tired argument and it seems to be parroted every single time someone talks about local models on hacker news. Yes, of course the most economical path is to hand over all your data and become fully dependent on a cloud provider who is already operating as scale, hoping that they won't change/remove models, hamstring capabilities, or raise prices. If this were a thread about hosting your own email or blo…
I don’t understand the willingness to give up privacy so easily, particularly if you are developing something that you plan to monetize somewhere down the road.
I’m pretty sure that all of those disclaimers that all the AI model makers have for you to sign off on to say that they’re not responsible for anything that might go wrong if your work gets copied accidentally and used someplace else wink wink?
You know the lawsuits for that particular aspect are incoming in the future…
Re: Apple caught off guard by AI demand for Mac Mini and Mac Studio
#320I get the impression they want AI marketing points but don’t actually want people to use local AI on their products.
I have no idea why. They could be so successful if they leaned into local AI.