Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Granite 4.1: IBM's 8B Model Matching 32B MoE

firethering.com

Post reply on HN
Print view View original

191–200 of 223 posts

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#191

Post by nickpsecurity »

Tuna-Fish wrote: post #188
nickpsecurity wrote: post #176

Earlier quoted context omitted.

What needs to happen is for companies (or individuals) tired of that to pool money together to build new, memory products. Then, sell them to consumers first and for non-AI use. If not that, then round-robin scheduling of quantities so the units are spread around more. If costs are high, they might reserve a certain percentage for big business at market prices (or just under) to cover the chip's mask costs. After DDR…

Time for my daily "HBF is coming" comment. The next step for models is to put the weights on flash, connected with a very wide interface to the accelerator. The first users will be datacenters, but it should trickle down to consumer hardware eventually. A single 512GB stack is expected to cost about $200, and provide 1.6TB/s of reads. You still need some fast DRAM for the KV cache and for activations, but weights sho…

You're thinking in a provably-useful direction:

https://arxiv.org/pdf/2312.11514

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#192

Post by zozbot234 »

Tuna-Fish wrote: post #188
nickpsecurity wrote: post #176

Earlier quoted context omitted.

What needs to happen is for companies (or individuals) tired of that to pool money together to build new, memory products. Then, sell them to consumers first and for non-AI use. If not that, then round-robin scheduling of quantities so the units are spread around more. If costs are high, they might reserve a certain percentage for big business at market prices (or just under) to cover the chip's mask costs. After DDR…

Time for my daily "HBF is coming" comment. The next step for models is to put the weights on flash, connected with a very wide interface to the accelerator. The first users will be datacenters, but it should trickle down to consumer hardware eventually. A single 512GB stack is expected to cost about $200, and provide 1.6TB/s of reads. You still need some fast DRAM for the KV cache and for activations, but weights sho…

Reading from Flash is too power-intensive compared to DRAM, this is why Flash offload isn't used in the data center today. Flash is also prone to wearing out quickly so ephemeral data like the KV-cache can't really be stashed in there. Unless your model has an unprecedented level of sparsity I just don't see how HBF could ever be useful.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#193

Post by bpye »

3abiton wrote: post #146
2ndorderthought wrote: post #4

I test drove it yesterday. It's pretty impressive at 8b. Runs on commodity hardware quickly. Qwen3.6 35b a3b is still my local champion but I may use this for auto complete and small tasks. Granite has recent training data which is nice. If the other small models got fine tuned on recent data I don't know if I would use this at all, but that alone makes it pretty decent. The 4b they released was not good for my needs…

> Qwen3.6 35b a3b is still my local champion but I may use this for auto complete and small tasks. I second this! Using the Unsloth Q6 (I forgot the exact name). Currently using it with forgecode (with zsh), on my Strix Halo, and it's suprisingly really good. I would say slightly Similar to Haiku 4.5, plus additional privacy, minus speed. It's surprisingly really fast for the hardware, given the speculative decoding,…

Out of interest, what are you seeing for token generation - especially as the context fills?

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#194

Post by EagnaIonat »

speedgoose wrote: post #177
EagnaIonat wrote: post #173

Earlier quoted context omitted.

You could have found in 5 seconds. The weights are also open sourced as well. https://github.com/ibm-granite

Maybe I suck but I didn’t find that in 5 seconds. Or with more time. I meant the full training datasets and the complete recipes to make the models.

The training datasets are listed there and are all open source.

> the complete recipes to make the models.

You mean the weights which most companies don't release. Again you can find from that link.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#195

Post by jeraldbenny »

speedgoose wrote: post #177
EagnaIonat wrote: post #173

Earlier quoted context omitted.

You could have found in 5 seconds. The weights are also open sourced as well. https://github.com/ibm-granite

Maybe I suck but I didn’t find that in 5 seconds. Or with more time. I meant the full training datasets and the complete recipes to make the models.

if you are googling you can find so many open source dataset. Also use kaggle, they're also having training datasets which we can use.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#196

Post by copper-float »

blurbleblurble wrote: post #158
v3ss0n wrote: post #51

Earlier quoted context omitted.

Qwen 3.6 burns it to the ground. it was not even a challenge. Gemma4 seriously fails at toolcalls and agentic works. It got all messed up after 2-3 turns of Vibecoding.

I agree but would add that gemma 4 is really nice at vibing though in ways qwen 3.6 could never. Maybe it could be fun to hook them up via a2a protocol as left and right brain agents operating in tandem.

As someone who has never used AI for any coding or agent tasks, I feel like i'm going insane when I read things like this.

>Maybe it could be fun to hook them up via a2a protocol as left and right brain agents operating in tandem

What in the world does this even mean?

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#197

Post by speedgoose »

EagnaIonat wrote: post #194
speedgoose wrote: post #177

Earlier quoted context omitted.

Maybe I suck but I didn’t find that in 5 seconds. Or with more time. I meant the full training datasets and the complete recipes to make the models.

The training datasets are listed there and are all open source. > the complete recipes to make the models. You mean the weights which most companies don't release. Again you can find from that link.

Where is the list?

No I didn't mean the weights, but the source code to make the weights.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#198

Post by speedgoose »

jeraldbenny wrote: post #195
speedgoose wrote: post #177

Earlier quoted context omitted.

Maybe I suck but I didn’t find that in 5 seconds. Or with more time. I meant the full training datasets and the complete recipes to make the models.

if you are googling you can find so many open source dataset. Also use kaggle, they're also having training datasets which we can use.

I know, but that is not my point.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#200

Post by adrian_b »

cbg0 wrote: post #11

The real "sleeper" might be https://huggingface.co/ibm-granite/granite-vision-4.1-4b if the benchmarks hold up for such a small model against frontier models for table & semantic k:v extraction.

There is also its companion:

https://huggingface.co/ibm-granite/granite-speech-4.1-2b

designed for multilingual automatic speech recognition (ASR) and bidirectional automatic speech translation (AST) for English, French, German, Spanish, Portuguese and Japanese.

Post reply on HN