[flagged]
Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
71–80 of 121 posts
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#72omlx gives better performance than ollama on apple silicon
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#73Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#74[dead]
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#75Local models are finally starting to feel pleasant instead of just "possible." The headless LM Studio flow is especially nice because it makes local inference usable from real tools instead of as a demo. Related note from someone building in this space: I've been working on cloclo ( https://www.npmjs.com/package/cloclo ), an open-source coding agent CLI, and this is exactly the direction I'm excited about. It nativel…
How does cloclo differ from pi-mono?
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#76[flagged]
[dead]
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#77Did you try the MLX model instead? In general MLX tends provide much better performance than GGUF/Llama.cpp on macOS.
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#78Earlier quoted context omitted.
Yes you're right technically, but in reality you'd be swapping them the (vast?) majority in and out per inference request so would create an enormous bottleneck for the use case the author is using for.
With unified memory, reading from RAM to GPU compute buffer is not that painful, and you can use partial RAM caching to minimize the impact of other kinds of swapping.
In practical terms, is this kind of architecture available to consumers except through Apple?
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#79Earlier quoted context omitted.
Only for chat sessions, not for agentic coding. It's just too slow to be practical (10 minutes to answer a simple question about a 2k LoC project - and that's with a 5070 addon card).
Doesn't the framework desktop have a Ryzen 395 AI? That's a unified memory architecture like the Macs.
That's discrete DDR5, it's not as fast as your regular VRAM.
Re: Running Gemma 4 locally with LM Studio's new headless CLI and Claude Code
#80Earlier quoted context omitted.
Even tried gemma4:31b and gemma4:31b with 128k context (I have 72GiB VRAM). Nothing. I'm cursed I guess. That's ollama-rocm if that matters (I had weird bugs on Vulkan, maybe gemma misbehaves on radeons somehow?..). UPD: tried ollama-vulkan. It works, gemma4:31b-it-q8_0 with 64k context!
The default context is 128k for the smaller Gemma 4’s and 256k for the bigger ones, so you’re cutting off context and it doesn’t know how to continue. Bump it to native (or -c 0 may work too)
In that case the model descriptor on ollama.com is incorrect, because it defaults to 16k. So I have to manually change that to 64/128k. I think you are talking about maximum context size.