Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I ran Gemma 4 as a local model in Codex CLI

blog.danielvaughan.com

Post reply on HN
Print view View original

81–90 of 123 posts

Re: I ran Gemma 4 as a local model in Codex CLI

#82

Post by jwr »

declan_roberts wrote: post #65
jwr wrote: post #27

Earlier quoted context omitted.

I do the same thing on a MacBook Pro with an M4 Max and 64GB. I had problems until the most recent LM Studio update (0.4.11+1), tool calling didn't work correctly. Now both codex and opencode seem to work.

Which do you prefer? And what lmstudio api works best for these tools?

I use the OpenAI API for everything. I think codex is more polished, but I don't really prefer anything: I haven't used them enough. I mostly use Claude Code.

Re: I ran Gemma 4 as a local model in Codex CLI

#84

Post by gertlabs »

Gemma 4 26B really is an outlier in its weight class.

In our little known, difficult to game benchmarks, it scored about as well as GPT 5.2 and Gemini 3 Pro Preview on one-shot coding problems. It had me re-reviewing our entire benchmarking methodology.

But it struggled in the other two sections of our benchmark: agentic coding and non-coding decision making. Tool use, iterative refinement, managing large contexts, and reasoning outside of coding brought the scores back down to reality. It actually performed worse when it had to use tools and a custom harness to write code for an eval vs getting the chance to one-shot it. No doubt it's been overfit on common harnesses and agentic benchmarks. But the main problem is likely scaling context on small models.

Still, incredible model, and incredible speed on an M-series Macbook. Benchmarks at https://gertlabs.com

Re: I ran Gemma 4 as a local model in Codex CLI

#85

Post by davidwritesbugs »

tuzemec wrote: post #9

I'm currently experimenting with running google/gemma-4-26b-a4b with lm studio ( https://lmstudio.ai/ ) and Opencode on a M3 Ultra with 48Gb RAM. And it seems to be working. I had to increase the context size to 65536 so the prompts from Opencode would work, but no other problems so far. I tried running the same on an M3 Max with less memory, but couldn't increase the context size enough to be useful with Opencode. I…

I did the same using the mlx version on an M1 Macbook using LMStudio integrated into XCode. I had to up the context size I ran it a against a very modest iOS codebase and it didn't do well, just petered out at one point. Odd. Pretty good chatbot and maybe against other code it'll work but not useful with XCode for me

Re: I ran Gemma 4 as a local model in Codex CLI

#86

Post by estimator7292 »

mapontosevenths wrote: post #54
dajonker wrote: post #18

I don't really have the hardware to try it out, but I'm curious to see how Qwen3.5 stacks up against Gemma 4 in a comparison like this. Especially this model that was fine tuned to be good at tool calling that has more than 500k downloads as of this moment: https://huggingface.co/Jackrong/Qwen3.5-27B-Claude-4.6-Opus-...

I'm just some guy on hackernews, but I actually did try this on my DGX Spark. I went back to Gemma 4 after a few rounds. My orchestration model kept having to send the Qwen model back to fix mistakes that Gemma wouldn't have made. I wound up with less working code per hour due to the mistakes. Technically, I use OpenWebUI with Ollama, so I used the weights below, but it should be the same. https://ollama.com/kwangsuk…

I'd be super interested to hear about your workflow with OpenWebUI. I haven't figured out how to use it for anything other than the basic chatbot UI. I haven't been able to hook anything else into it

Re: I ran Gemma 4 as a local model in Codex CLI

#87

Post by mapontosevenths »

estimator7292 wrote: post #86
mapontosevenths wrote: post #54

Earlier quoted context omitted.

I'm just some guy on hackernews, but I actually did try this on my DGX Spark. I went back to Gemma 4 after a few rounds. My orchestration model kept having to send the Qwen model back to fix mistakes that Gemma wouldn't have made. I wound up with less working code per hour due to the mistakes. Technically, I use OpenWebUI with Ollama, so I used the weights below, but it should be the same. https://ollama.com/kwangsuk…

I'd be super interested to hear about your workflow with OpenWebUI. I haven't figured out how to use it for anything other than the basic chatbot UI. I haven't been able to hook anything else into it

What I said above was a bit confused. What I've actually done is connect OpenCode and OpenWebUI both to Ollama. I just use OpenWebUI to manage the models and for testing/etc. Once you have it working it's very nice. You can pull a new model just by typing the name and waiting while it downloads, etc.

Connecting Ollama to OpenCode and OpenWebUI is relatively trivial. In OpenWebUI there's a nice GUI. In OpenCode You just edit the ~/.config/opencode/opecode.json to look something like this. The model names have to match the ones you seen in OpenWebUI, but the friendly "name" key can be whatever you need to be able to recognize it.

  {
    "$schema": "https://opencode.ai/config.json",
    "provider": {
   "ollama": {
     "npm": "@ai-sdk/openai-compatible",
     "name": "Ollama",
     "options": {
    "baseURL": "http://localhost:11434/v1"
     },
     "models": {
    "qwen3.5:122b": {
      "name": "Qwen 3.5 122b"
    },
    "qwen3-coder:30b": {
      "name": "Qwen 3 Coder"
    },
    "gemma4:26b": {
      "name": "Gemma 4"
    }
     }
   }
    }
  }

Re: I ran Gemma 4 as a local model in Codex CLI

#88

Post by 59nadir »

59nadir wrote: post #46
ehtbanton wrote: post #5

This is genuinely very helpful. I'm planning a MacBook pro purchase with local inference in mind and now see I'll have to aim for a slightly higher memory option because the Gemma A4 26B MoE is not all that!

If you're doing it specifically for inference (or in most other situations) a Mac(book) represents very low RoE.

s/RoE/RoI

Re: I ran Gemma 4 as a local model in Codex CLI

#90

Post by tuzemec »

usagisushi wrote: post #69
tuzemec wrote: post #9

I'm currently experimenting with running google/gemma-4-26b-a4b with lm studio ( https://lmstudio.ai/ ) and Opencode on a M3 Ultra with 48Gb RAM. And it seems to be working. I had to increase the context size to 65536 so the prompts from Opencode would work, but no other problems so far. I tried running the same on an M3 Max with less memory, but couldn't increase the context size enough to be useful with Opencode. I…

I have a similar setup. It might be worth checking out pi-coding-agent [0]. The system prompt and tools have very little overhead ( [0] https://www.npmjs.com/package/@mariozechner/pi-coding-agent#...

Thanks! I just ran a quick test with pi, and it's working a bit faster.
Post reply on HN