My local model setup on an M4 Pro Mac Mini

Post by whatsThisBtn4 » Sat, Sep 05, 2026, 11:27 PM UTC

EagnaIonat wrote: post #193

whatsThisBtn4 wrote: post #188

Earlier quoted context omitted.

You could buy a $150 refurbished 16gb i5 and use that model until openAI does its IPO and has to become sane again. But I guess a $2000 Mac is probably better if you don't care about cost or quality.

> You could buy a $150 refurbished 16gb i5 Or I can use the Mac I already have. Your example though, Ouch! ~8B Q4. That's around 5-10 tokens a second. Base M1 16GB mac would do 15-20 tokens a seconds. That's a 6 year old machine. You do get what you pay for it seems.

Post by dnugget » Wed, Sep 09, 2026, 6:51 PM UTC

mintflow wrote: post #10

Have a macmini m4 32G, not the pro version, previously everytime I tried local LLM is a bit disappointing, and I finally decide to not waste time and perhaps in the future invest a better hardware to server more modern and dense model I am curious is what is the 80% request served by this setup, I was using it for OpenClaw which run serveral cron jobs that discover stuffs over the wide internet, check my support syst…