Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Granite 4.1: IBM's 8B Model Matching 32B MoE

firethering.com

Post reply on HN
Print view View original

81–90 of 223 posts

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#81

Post by 2ndorderthought »

throwaw12 wrote: post #63
2ndorderthought wrote: post #42

Earlier quoted context omitted.

I tried the Gemma 4 I think 2 and 4b. The 2b was not useful for me at all. A little too weak for my use cases The 4b was okay. It didn't get all of my small math questions right, it didn't know about some of the libraries I use, but it was able to do some basic auto complete type stuff. For microscopic models I like the llama 3.2 3b more right now for what I do, it's a little faster and seems a little stronger for wh…

can you share your use cases for 2b and 4b models? curious how people are leveraging these models

For me, I use them for quick auto complete or small questions. I am not a vibe/agentic coder. I know I am a relic and a Luddite because of this.

Instead of hitting stack overflow and Google I will ask questions like "can you give me an example of how to do x in library y?" Or "this error is appearing what might be happening if I checked a b and c". Or "please write unit tests for this function". Or code auto complete.

I am not looking for the world's best answer from a 3b model. I am looking for a super fast answer that reminds me of things I already know or maybe just maybe gives me a fast idea to stub something while I focus on something more important, I am going to refactor anyways. Think a low quality rubber duck

I mostly use 7-9b models for this now but llama 3.2 3b is pretty decent for not hogging resources while say I have other compute heavy operations happening on a weak computer.

Probably half the questions people ask chatgpt could get roughly the same quality of answer with a small model in my opinion. You can't fully trust an LLM anyways so the difference between 60% and 70% accuracy isn't as much are marketing makes it sound like. That said the quality of a good 7-9b model is worth it compared to a 3b if your machine can run it. Furthermore the quality of qwen 36 is crazy and makes me wonder if I will ever need an AI provider again if the trend continues.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#82

Post by 0xbadcafebee »

People complain a lot about LLM-written articles, but the human comments here on HN are far worse. Mostly a bunch of people extremely proud of themselves for not reading an LLM-written article, and then a bunch of people who take it at face value and make the model seem almost useful, and one comment that actually looked at other benchmarks. Good 'ol humanity, good at.. being emotional... and not doing analysis.....

The article makes some good points about model design (how different size models within a family can get similar results, how to filter out hallucination, math result reinforcement), so that's worth understanding. It's analyzing a paper, which only discussed 3 sizes of the same model family. But what the article doesn't say is, compared to other model families, Granite 4.1 8B sucks. The only benchmark it does well at compared to other models is non-hallucination and instruction following. Qwen 3.5 4B (among other models) easily outclass it on every other metric.

This article teaches a valuable lesson about reading articles in general. You can take useful information away from them (yes, despite being written by LLM). But you should also use critical thinking skills and be proactive to see if the article missed anything you might find relevant.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#83

Post by bityard »

wewewedxfgdf wrote: post #14
100ms wrote: post #9

> Full stop. Why people don't edit out obvious sloppification and expect to still have readers left

Third line in to the article: "But there’s one result in the benchmarks I keep coming back to." I hear this sort of thing all the time now on YouTube from media/news personalities: “And that’s the part nobody seems to be talking about.” "And here's what keeps me up at night." “This is where the story gets complicated.” “Here’s the piece that doesn’t quite fit.” “And this is where the usual explanation starts to break…

I listened to a lot of NPR podcasts before LLM were around, and most of them are full of these kinds of filler phrases.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#84

Post by osener »

dash2 wrote: post #67

Nah, I ain't reading that. If they can't be bothered to get a human to write it, it can't be that important. I'm glad for them though. Or sorry that happened.

This is the official announcement: https://research.ibm.com/blog/granite-4-1-ai-foundation-mode...

It is not the researchers' fault that some slop got posted here instead.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#85

Post by helsinkiandrew »

wewewedxfgdf wrote: post #14
100ms wrote: post #9

> Full stop. Why people don't edit out obvious sloppification and expect to still have readers left

Third line in to the article: "But there’s one result in the benchmarks I keep coming back to." I hear this sort of thing all the time now on YouTube from media/news personalities: “And that’s the part nobody seems to be talking about.” "And here's what keeps me up at night." “This is where the story gets complicated.” “Here’s the piece that doesn’t quite fit.” “And this is where the usual explanation starts to break…

Isn't this the format of "hook-driven media" a constant stream of "second-act pivots" - where some new twist is added to a story to re-engage the reader and keep them reading.

BuzzFeed and Upworthy etc pioneered this for web 'news stories', then it got used in linkedin, twitter, and everywhere where views are more important than the content.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#86

Post by Lalabadie »

2ndorderthought wrote: post #64
uf00lme wrote: post #57

Earlier quoted context omitted.

Woah, is this part of the future of models? Basically little models you can use as tools.

It's looking like running your own mini ecosystem is the way of the future to me. No data centers, just a decent GPU 16-24gb of VRAM, CPU, and 32gb of RAM.

This is Apple's bet, among others.

Training purpose-specific miniature models lets you have a lot of tasks you can run with high confidence on consumer hardware.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#87

Post by 59nadir »

v3ss0n wrote: post #51
vessenes wrote: post #32

Earlier quoted context omitted.

Have you tried the Gemma 4 series, out of curiosity? I haven’t run a local model in a while, but the benchmarks look good. I’d take a free local tool-use model if it was relatively consistent.

Qwen 3.6 burns it to the ground. it was not even a challenge. Gemma4 seriously fails at toolcalls and agentic works. It got all messed up after 2-3 turns of Vibecoding.

Counter-point: I built an agent that can only interface with Kakoune, a much less common and more challenging situation for an LLM to find itself in, and Gemma4-A4B 8bit quantized does remarkably better in actually figuring out how to get text in buffers than Qwen3.6-35B-A3B in a similar class as Gemma4 A4B.

Now, is this the usual use case? No, it's a benchmark I created specifically in order to put LLMs in situations where they can't just blast out their bash commands without having to interface with something else and adapt.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#88

Post by regularfry »

zozbot234 wrote: post #25
agunapal wrote: post #18

If you really think about why MoE came into existence, its to save significant cost during training, I don't think there was any concrete evidence of performance gains for comparable MoE vs dense models. Over the years, I believe all the new techniques being employed in post training have made the models better.

MoE models will have far more world knowledge than dense models with the same amount of active parameters. MoE is a no-brainer if your inference setup is ultimately limited by compute or memory throughput - not total memory footprint - or alternately if it has fast, high-bandwidth access to lower-tier storage to fetch cold model weights from on demand.

Yes, this. I can run the 122B Qwen3.5 MoE usably on one 4090 + 64GB RAM. That's a monster of a model, comparatively speaking.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#89

Post by twoodfin »

Lalabadie wrote: post #86
2ndorderthought wrote: post #64

Earlier quoted context omitted.

It's looking like running your own mini ecosystem is the way of the future to me. No data centers, just a decent GPU 16-24gb of VRAM, CPU, and 32gb of RAM.

This is Apple's bet, among others. Training purpose-specific miniature models lets you have a lot of tasks you can run with high confidence on consumer hardware.

Or on a commodity EC2 instance with a relatively cheap inference sidecar.
Post reply on HN