Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Granite 4.1: IBM's 8B Model Matching 32B MoE

firethering.com

Post reply on HN
Print view View original

51–60 of 223 posts

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#51

Post by v3ss0n »

vessenes wrote: post #32
2ndorderthought wrote: post #4

I test drove it yesterday. It's pretty impressive at 8b. Runs on commodity hardware quickly. Qwen3.6 35b a3b is still my local champion but I may use this for auto complete and small tasks. Granite has recent training data which is nice. If the other small models got fine tuned on recent data I don't know if I would use this at all, but that alone makes it pretty decent. The 4b they released was not good for my needs…

Have you tried the Gemma 4 series, out of curiosity? I haven’t run a local model in a while, but the benchmarks look good. I’d take a free local tool-use model if it was relatively consistent.

Qwen 3.6 burns it to the ground. it was not even a challenge. Gemma4 seriously fails at toolcalls and agentic works. It got all messed up after 2-3 turns of Vibecoding.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#52

Post by Havoc »

npodbielski wrote: post #50
Havoc wrote: post #5

Interesting to see a pivot away from MoE by both IBM and mistral while the larger classes of SOTA of models all seem to be sticking to it. Quick vibe check of it- 8B @ Q6 - seems promising. Bit of a clinical tone, but can see that being useful for data processing and similar. You don't really want a LLM that spams you with emojis sometimes...

I never want LLM to span me with emojis. What is the use case for that? I find it highly annoying.

Think it can be a plus in moderation. eg in openclaw it can add some character

But yea dislike that style where each heading and bullet point gets an emoji

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#53

Post by Terretta »

irishcoffee wrote: post #44
noodletheworld wrote: post #39

Earlier quoted context omitted.

Having tried it. Qwen is really good. Also, generally, it makes sense. 8B models are generally not very good^. That this 8B model is decent is impressive, but that it could perform on par with a good model 4 times as large is a daydream. ^ - To be polite. The small models + tool use for coding agents are almost universally ass. Proof: my personal experience. Ive tried many of them.

So it’s just like, your opinion, man? edit: It was a play on The Big Lebowski, folks.

College SAT scores do not tell you how the dev applying for your open back end systems engineering job is going to do once they're in your workplace harness.

Nor do class standings, nor hackerrank and the like.

What will tell you is asking them to fix a thing in your codebase. Once you ask an LLM to do that, a dozen times, I'd argue it's no longer "just your opinion man", it's a context-engineered performance x applicability assessment.

And it is very predictive.

But it's also why someone doing well at job A isn't necessarily going to be great at B, or bad at A doesn't mean will necessarily be bad at B.

I've often felt we should normalize a sort of mutual try-buy period where job-change seeker and company can spend a series of days without harming one's existing employment, to derisk the mutual learning. ESPECIALLY to derisk the career change for the applicant who only gets one timeline to manage, opposed to company that considers the applicant fungible.

But back to the LLM, yeah, the only valid opinion on whether it works for you is not benchmark, it's an informed opinion from 'using it in anger'.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#55

Post by bjourne »

vessenes wrote: post #28
agunapal wrote: post #18

If you really think about why MoE came into existence, its to save significant cost during training, I don't think there was any concrete evidence of performance gains for comparable MoE vs dense models. Over the years, I believe all the new techniques being employed in post training have made the models better.

I think you mean inference compute? I believe all expert weights are updated in each backward pass during MoE training. The first benefit was getting a sort of structured pruning of weights through the mechanism of expert selection so that the model didn’t need to go through ‘unnecessary’ parts of the model for a given token. This then let inference use memory more efficiently in memory constrained environments, wher…

Each token is only routed through a few chosen (topk) experts during training. So not all expert weights are updated in the backward pass. Otoh, you may need more training to ensure all experts see enough tokens!

I doubt MoE is actually worth it, given how complicated high-performance expert routing and training is. But who knows, I don't.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#57

Post by uf00lme »

cbg0 wrote: post #11

The real "sleeper" might be https://huggingface.co/ibm-granite/granite-vision-4.1-4b if the benchmarks hold up for such a small model against frontier models for table & semantic k:v extraction.

Woah, is this part of the future of models? Basically little models you can use as tools.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#59

Post by 2ndorderthought »

v3ss0n wrote: post #51
vessenes wrote: post #32

Earlier quoted context omitted.

Have you tried the Gemma 4 series, out of curiosity? I haven’t run a local model in a while, but the benchmarks look good. I’d take a free local tool-use model if it was relatively consistent.

Qwen 3.6 burns it to the ground. it was not even a challenge. Gemma4 seriously fails at toolcalls and agentic works. It got all messed up after 2-3 turns of Vibecoding.

Gemma4 is definitely not used for vibe/agentic coding. Not even worth trying. But its a different weight class.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#60

Post by cyanydeez »

2ndorderthought wrote: post #4

I test drove it yesterday. It's pretty impressive at 8b. Runs on commodity hardware quickly. Qwen3.6 35b a3b is still my local champion but I may use this for auto complete and small tasks. Granite has recent training data which is nice. If the other small models got fine tuned on recent data I don't know if I would use this at all, but that alone makes it pretty decent. The 4b they released was not good for my needs…

Qwen3-Coder-Next seems to be perfect sized for coding. I tried the new and just found the verbosity not really useful for coding. But probably for more analytical tasks or writing docs.
Post reply on HN