Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Anyone Using a Mac Studio for Local AI/LLM?

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 39 posts

Re: Ask HN: Anyone Using a Mac Studio for Local AI/LLM?

#11

Post by StrangeSound »

giancarlostoro wrote: post #5

Not a Mac Studio but I use a basic Macbook Pro laptop with 24 GB of RAM (16 usable as VRAM) and I can run a number of models on it at decent speed, my main bottleneck is context window size, but if I am asking single purpose questions I am fine.

What models are you running?

Re: Ask HN: Anyone Using a Mac Studio for Local AI/LLM?

#13

Post by circularfoyers »

StevenNunez wrote: post #3

I do! I have an M3 Ultra with 512GB. A couple of opencode sessions running work well. Currently running GML 4.7 but was on Kimi K2.5. Both great. Excited for more efficiencies to make their way to LLMs in general.

The prompt processing times I've heard about have put me off wanting to go that high with memory on the M series (hoping that changes for the M5 series though). What's the average and longest times you've had to wait when using opencode? Has any improvements to mlx helped in that regard?

Re: Ask HN: Anyone Using a Mac Studio for Local AI/LLM?

#15

Post by ryan-c »

I'm using an M3 Ultra w/ 512GB of RAM, using LMStudio and mostly mlx models. It runs massive models with reasonable tokens per second, though prompt processing can be slow. It handles long conversations fine so long as the KV cache hits. It's usable with opencode and crush, though my main motivation for getting it was specifically to be able to process personal data (e.g. emails) privately, and to experiment freely with abliterated models for security research. Also, I appreciate being able to run it off solar power.

I'm still trying to figure out a good solution for fast external storage, I only went for 1TB internal which doesn't go very far with models that have hundreds of billions of parameters.

Re: Ask HN: Anyone Using a Mac Studio for Local AI/LLM?

#16

Post by rlupi »

I have an M3 Ultra 96 GB, it works reasonably well with something like qwen/qwen3-vl-30b (fast) or openai/gpt-oss-120b (slow-ish) or openai/gpt-oss-20b (fast, largest context). I keep the latter loaded, and have a cronjob that generates a new MOTD for my shell every 15 minutes with information gathered from various sources.

Re: Ask HN: Anyone Using a Mac Studio for Local AI/LLM?

#20

Post by runjake »

For anything other than a toy, I would recommend at least a Max processor and at least 32 GB memory, depending on what you're doing. I do a lot of text, audio, and NLP stuff, so I'm running smaller models and my 36GB is plenty.

Ultra processors are priced high enough, I'd be asking myself if I'm serious about local LLM work and do a cost analysis.

Post reply on HN