61–70 of 150 posts
I wrote a similar post some time ago just used ollama and opencode https://blog.kulman.sk/running-local-llm-coding-server/
this is the way, given anyone could swap for oh my pi / pi / etc
Useful stuff in here that I wish I'd seen a few days ago :-) I am not convinced that the MTP setup for the QAT model adds very much in terms of speed on my M1 Max, but it is definitely worth experimenting with. Fiddling about with local models has done so much for my conceptual understanding of what is going on. FWIW and YMMV but I also found the Gemma 4 MTP head was occasionally breaking markup in Opencode, causing…
>64 GB Thats the rub. I have an M4 with 48G. I wonder if it is worth testing this out. My past attempts (with Ollama and various LLMs) were too slow to use.
I have a M5 MAX with 128, local models are toys compared to hosted ones. I've spent a lot of time and money trying to make it work even 1/2 as well.
>64 GB Thats the rub. I have an M4 with 48G. I wonder if it is worth testing this out. My past attempts (with Ollama and various LLMs) were too slow to use.
I'm running an M3 on an Air with just 16GB. I can still get useful results without an internet connection in "chat mode". It's a different experience than using Claude, for sure, but it's workable. I typically use the Qwen variants these days.
Earlier quoted context omitted.
I'm running an M3 on an Air with just 16GB. I can still get useful results without an internet connection in "chat mode". It's a different experience than using Claude, for sure, but it's workable. I typically use the Qwen variants these days.
This might be useful when ‘coding in chat mode’: I have a few scripts that I run in a project directory that takes a prompt from me, and creates a single long one-shot prompt that I can paste into a chat window and ask that any generating code is inside markdown code blocks for easier copy/pasting. Also, pardon the plug, but you can read my new tiny book free online that documents my experiences using agentic coding…
> NoSuchKeyThe specified key does not exist…
My biggest pet peeve with all these articles on local AI is the only thing they talk about is tokens per second. No one mentions the quality of the answers. No one. I don't mind waiting a little longer if the quality is better. Quickly serving me slop doesn't make it more useful. Are people really only looking at tokens per second?
So there is no value in testing quality of answers, but there is value in testing token speed.
You just have to have correct expectations.
I assumed lmstudio is the obvious choice after ollama. Is there a reason lmstudio is not used widely ?
Earlier quoted context omitted.
Yeah, if the future is "Claude, think for me" I'm happy to stay at the good old present.
https://en.wikipedia.org/wiki/Is_Google_Making_Us_Stupid%3F https://newsletter.pessimistsarchive.org/p/when-educators-mo... New decade, same old argument. It's not > "Claude, think for me" It's > "Claude, be my subordinate and get this done for me" Instead of complaining on the sidelines, I'm getting a shit ton of work done.
An argument can be as old as the search engine and hold real value. There are ways in which unreflective search engine use has misled and mistrained people.
There’s always been argument to be had about how we manage and offload attention, what we gain and what we lose when resistance is reduced. It’s part of reflection that’s been necessary in order to make progress solid ground, and is more necessary with non-deterministic tech.
The phrase “Tactical tornados” may be older than web search and describes people who also got a lot done.
Models can be incredibly helpful boosters and situationally effective subordinates… and also patchy as a real engineering IC or org.
harbor up omlx opencode