Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code

ai.georgeliu.com

Post reply on HN
Print view View original

51–60 of 121 posts

Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code

#53

Post by charcircuit »

martinald wrote: post #32
charcircuit wrote: post #20

Earlier quoted context omitted.

You never need to have all weights in memory. You can swap them in from RAM, disk, the network, etc. MOE reduces the amount of data that will need to be swapped in for the next forward pass.

Yes you're right technically, but in reality you'd be swapping them the (vast?) majority in and out per inference request so would create an enormous bottleneck for the use case the author is using for.

You don't have to only have the experts being actively used in VRAM. You can load as many weights as will fit. If there is a "cache miss" you have to pay the price to swap in the weights, but if there is a hit you don't.

Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code

#54

Post by tarruda »

loveparade wrote: post #49
NamlchakKhandro wrote: post #47

I don't know why people bother with Claude code. It's so jank, there are far superior cli coding harness out there

What do you recommend? I've tried both pi and opencode and both are better than claude imo, but I wonder if there are others.

Codex is the best out-of-box experience, especially due to its builtin sandboxing. Only drawback is that its edit tool requires the LLM to output a diff which only GPTs are trained to do correctly.

Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code

#56

Post by loveparade »

tarruda wrote: post #54
loveparade wrote: post #49

Earlier quoted context omitted.

What do you recommend? I've tried both pi and opencode and both are better than claude imo, but I wonder if there are others.

Codex is the best out-of-box experience, especially due to its builtin sandboxing. Only drawback is that its edit tool requires the LLM to output a diff which only GPTs are trained to do correctly.

Interesting, I don't like codex exactly because of its built-in sandboxing. If I need a sandbox I rather do a simple bwrap myself around the agent process, I prefer that over the agent cli doing a bunch of sandboxing magic that gets in my way.

Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code

#57

Post by mil22 »

gcampos wrote: post #55
trvz wrote: post #4

ollama launch claude --model gemma4:26b

You need to increase the context window size or the tool calling feature wont work

For those wondering how to do this:

  OLLAMA_CONTEXT_LENGTH=64000 ollama serve
or if you're using the app, open the Ollama app's Settings dialog and adjust there.

Codex also works:

  ollama launch codex --model gemma4:26b

Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code

#58

Post by satvikpendem »

unsnap_biceps wrote: post #14
keerthiko wrote: post #9

Earlier quoted context omitted.

Claude Code is fairly notoriously token inefficient as far as coding agent/harnesses go (i come from aider pre-CC). It's only viable because the Max subscriptions give you approximately unlimited token budget, which resets in a few hours even if you hit the limit. But this also only works because cloud models have massive token windows (1M tokens on opus right now) which is a bit difficult to make happen locally with…

I use CC at work, so I haven't explored other options. Is there a better one to use locally? I presumed they were all going to be pretty similar.

OpenCode

Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code

#59

Post by prettyblocks »

tarruda wrote: post #54
loveparade wrote: post #49

Earlier quoted context omitted.

What do you recommend? I've tried both pi and opencode and both are better than claude imo, but I wonder if there are others.

Codex is the best out-of-box experience, especially due to its builtin sandboxing. Only drawback is that its edit tool requires the LLM to output a diff which only GPTs are trained to do correctly.

how is codex sandbox different from /sandbox on claude code?
Post reply on HN