131–140 of 223 posts
> Reduce your expectations about speed and performance! Wildly understating this part. Even the best local models (ones you run on beefy 128GB+ RAM machines) get nowhere close to the sheer intelligence of Claude/Gemini/Codex. At worst these models will move you backwards and just increase the amount of work Claude has to do when your limits reset.
Earlier quoted context omitted.
The best open models such as Kimi 2.5 are about as smart today as the big proprietary models were one year ago. That's not "nothing" and is plenty good enough for everyday work.
The article mentions https://unsloth.ai/docs/basics/claude-codex I'll add on https://unsloth.ai/docs/models/qwen3-coder-next The full model is supposedly comparable to Sonnet 4.5 But, you can run the 4 bit quant on consumer hardware as long as your RAM + VRAM has room to hold 46GB. 8 bit needs 85.
Earlier quoted context omitted.
The best open models such as Kimi 2.5 are about as smart today as the big proprietary models were one year ago. That's not "nothing" and is plenty good enough for everyday work.
Which takes a $20k thunderbolt cluster of 2 512GB RAM Mac Studio Ultras to run at full quality…
https://buildai.substack.com/i/181542049/the-mac-mini-moment
Useful tip. From a strategic standpoint of privacy, cost and control, I immediately went for local models, because that allowed to baseline tradeoffs and it also made it easier to understand where vendor lock-in could happen, or not get too narrow in perspective (e.g. llama.cpp/open router depending on local/cloud [1] ). With the explosion of popularity of CLI tools (claude/continue/codex/kiro/etc) it still makes sen…
can you recommend a setup with ollama and a cli tool? Do you know if I need a licence for Claude if I only use my own local LLM?
I also highly suggest OpenCode. You'll get the same Claude Code vibe.
If your computer is not beefy enough to run them locally, Synthetic is a bless when it comes to providing these models, their team is responsive, no downtime or any issue for the last 6 months.
Full list of models provided : https://dev.synthetic.new/docs/api/models
Referal link if you're interested in trying it for free, and discount for the first month : https://synthetic.new/?referral=kwjqga9QYoUgpZV
Earlier quoted context omitted.
Yeah this is why I ended up getting Claude subscription in the first place. I was using GLM on ZAI coding plan (jerry rigged Claude Code for $3/month), but finding myself asking Sonnet to rewrite 90% of the code GLM was giving me. At some point I was like "what the hell am I doing" and just switched. To clarify, the code I was getting before mostly worked, it was just a lot less pleasant to look at and work with. Mig…
> but finding myself asking Sonnet to rewrite 90% of the code GLM was giving me. At some point I was like "what the hell am I doing" and just switched. This is a very common sequence of events. The frontier hosted models are so much better than everything else that it's not worth messing around with anything lesser if doing this professionally. The $20/month plans go a long way if context is managed carefully. For a…
[flagged]
[flagged]
Obviously it must be assumed that the model one falls back on is good enough - including security alignment.
Has anyone had a better experience?
Earlier quoted context omitted.
Correct, a rack full of datacenter equipment is not going to compete with anything that fits on your desk or lap. Well spotted. But as a counterpoint: there are whole communities of people in this space who get significant value from models they run locally. I am one of them.
What do you use local models for? I'm asking generally about possible applications of these smaller models
If you’re worried about others being able to clone your business processes if you share them with a frontier provider then the cost of a Mac Studio to run Kimi is probably a justifiable tax right off.