Earlier quoted context omitted.
You could buy a $150 refurbished 16gb i5 and use that model until openAI does its IPO and has to become sane again. But I guess a $2000 Mac is probably better if you don't care about cost or quality.
> You could buy a $150 refurbished 16gb i5 Or I can use the Mac I already have. Your example though, Ouch! ~8B Q4. That's around 5-10 tokens a second. Base M1 16GB mac would do 15-20 tokens a seconds. That's a 6 year old machine. You do get what you pay for it seems.
Oh go get a $700 Nvidia laptop.
Sorry local models are basically useless outside chat, I didn't even consider it.