ollama launch claude --model gemma4:26b
For some reason, that doesn't work for me, claude never returns from some ill loop. Nemotron, glm and qwen 3.5 work just fine, gemma - doesn't.
Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
21–30 of 121 posts
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#22[flagged]
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#23Earlier quoted context omitted.
Claude Code is fairly notoriously token inefficient as far as coding agent/harnesses go (i come from aider pre-CC). It's only viable because the Max subscriptions give you approximately unlimited token budget, which resets in a few hours even if you hit the limit. But this also only works because cloud models have massive token windows (1M tokens on opus right now) which is a bit difficult to make happen locally with…
I use CC at work, so I haven't explored other options. Is there a better one to use locally? I presumed they were all going to be pretty similar.
If you want to experiment with same-harness-different-models Opencode is classically the one to use. After their recent kerfluffle with Anthropic you'll have to use API pricing for opus/sonnet/haiku which makes it kind of a non-starter, but it lets you swap out any number of cloud or local models using e.g. ollama or z.ai or whatever backend provider you like.
I'd rate their coding agent harness as slightly to significantly less capable than claude code, but it also plays better with alternate models.
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#24Just FYI, MoE doesn't really save (V)RAM. You still need all weights loaded in memory, it just means you consult less per forward pass. So it improves tok/s but not vram usage.
It does if you use an inference engine where you can offload some of the experts from VRAM to CPU RAM. That means I can fit a 35 billion param MoE in let's say 12 GB VRAM GPU + 16 gigs of memory.
With that you are taking a significant performance penalty and become severely I/O bottlenecked. I've been able to stream Qwen3.5-397B-A17B from my M5 Max (12 GB/s SSD Read) using the Flash MoE technique at the brisk pace of 10 tokens per second. As tokens are generated different experts need to be consulted resulting in a lot of I/O churn. So while feasible it's only great for batch jobs not interactive usage.
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#25So wait what is the interaction between Gemma and Claude?
lm studio offers an Anthropic compatible local endpoint, so you can point Claude code at it and it'll use your local model for it's requests, however, I've had a lot of problems with LM Studio and Claude code losing it's place. It'll think for awhile, come up with a plan, start to do it and then just halt in the middle. I'll ask it to continue and it'll do a small change and get stuck again. Using ollama's api doesn'…
I don't get why I would use Claude Code when OpenCode, Cursor, Zed, etc. all exist, are "free" and work with virtually any llm. Seems like a weird use case unless I'm missing something.
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#26ollama launch claude --model gemma4:26b
For some reason, that doesn't work for me, claude never returns from some ill loop. Nemotron, glm and qwen 3.5 work just fine, gemma - doesn't.
Since that defaults to the q4 variant, try the q8 one:
ollama launch claude --model gemma4:26b-a4b-it-q8_0Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#27Earlier quoted context omitted.
lm studio offers an Anthropic compatible local endpoint, so you can point Claude code at it and it'll use your local model for it's requests, however, I've had a lot of problems with LM Studio and Claude code losing it's place. It'll think for awhile, come up with a plan, start to do it and then just halt in the middle. I'll ask it to continue and it'll do a small change and get stuck again. Using ollama's api doesn'…
I don't get why I would use Claude Code when OpenCode, Cursor, Zed, etc. all exist, are "free" and work with virtually any llm. Seems like a weird use case unless I'm missing something.
this is like asking why use intellij or vscode or … when there is vim and emacs
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#28Is a framework desktop with >48GB of RAM a good machine to try this out?
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#29Earlier quoted context omitted.
lm studio offers an Anthropic compatible local endpoint, so you can point Claude code at it and it'll use your local model for it's requests, however, I've had a lot of problems with LM Studio and Claude code losing it's place. It'll think for awhile, come up with a plan, start to do it and then just halt in the middle. I'll ask it to continue and it'll do a small change and get stuck again. Using ollama's api doesn'…
Claude Code is fairly notoriously token inefficient as far as coding agent/harnesses go (i come from aider pre-CC). It's only viable because the Max subscriptions give you approximately unlimited token budget, which resets in a few hours even if you hit the limit. But this also only works because cloud models have massive token windows (1M tokens on opus right now) which is a bit difficult to make happen locally with…
[dead]
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#30Using Claude Code seems like a popular frontend currently, I wonder how long until Anthropic releases an update to make it a little to a lot less turn-key? They've been very clear that they aren't exactly champions of this stuff being used outside of very specific ways.
Well, if they did, it would probably be shooting themselves in the foot, seeing that the Claude Code source is out there now, and people are waiting for an excuse to "clean-room" reimplement and fork it