> models are judged by GPT-4
An interesting choice
Granite 4.1: IBM's 8B Model Matching 32B MoE
71–80 of 223 posts
Re: Granite 4.1: IBM's 8B Model Matching 32B MoE
#72Re: Granite 4.1: IBM's 8B Model Matching 32B MoE
#73https://research.ibm.com/blog/granite-4-1-ai-foundation-mode...
Original article on IBM research
Hugging face weights: https://huggingface.co/collections/ibm-granite/granite-41-la...
Re: Granite 4.1: IBM's 8B Model Matching 32B MoE
#74Very impressive series of SLM by IBM here.
I have been using it with their Chunkless RAG concept and it is fitting very well! (for curious https://github.com/scub-france/Docling-Studio)
I convinced that SLM are a real parto of solution for true integrated AI in process...
Re: Granite 4.1: IBM's 8B Model Matching 32B MoE
#75Earlier quoted context omitted.
Have you tried the Gemma 4 series, out of curiosity? I haven’t run a local model in a while, but the benchmarks look good. I’d take a free local tool-use model if it was relatively consistent.
Qwen 3.6 burns it to the ground. it was not even a challenge. Gemma4 seriously fails at toolcalls and agentic works. It got all messed up after 2-3 turns of Vibecoding.
How do you run it? vllm? llama.cpp?
Can you share some parameters you enable tool calling and agentic usage?
Or, higher level, some philosophies on what approaches you are using for tuning to get better tool calling and/or agentic usage?
I'm having surprisingly good success with unsloth/Qwen3.6-27B-GGUF:Q4_K_M (love unsloth guys) on my RTX3090/24GB using opencode as the orchestrator.
It concocts some misleading paths, but the code often compiles, and I consider that a victory.
You have to watch it like you would watch a 14 year old boy who says he is doing his homework but you hear the sound effects of explosions.
Re: Granite 4.1: IBM's 8B Model Matching 32B MoE
#76Earlier quoted context omitted.
Have you tried the Gemma 4 series, out of curiosity? I haven’t run a local model in a while, but the benchmarks look good. I’d take a free local tool-use model if it was relatively consistent.
Qwen 3.6 burns it to the ground. it was not even a challenge. Gemma4 seriously fails at toolcalls and agentic works. It got all messed up after 2-3 turns of Vibecoding.
Gemma 4 31b was working ok for me; but it was consuming tons of memory on SWA checkpoints, I had to turn them way down, and as a 31b dense model is fairly slow on a Strix Halo. I did have a lot of tool calling issues on 26b-a4b, though.
The Qwen models are quite solid though.
Re: Granite 4.1: IBM's 8B Model Matching 32B MoE
#77Re: Granite 4.1: IBM's 8B Model Matching 32B MoE
#78Nah, I ain't reading that. If they can't be bothered to get a human to write it, it can't be that important. I'm glad for them though. Or sorry that happened.
[dead]
Re: Granite 4.1: IBM's 8B Model Matching 32B MoE
#79Nah, I ain't reading that. If they can't be bothered to get a human to write it, it can't be that important. I'm glad for them though. Or sorry that happened.
[deleted]