Live data from Hacker News

Granite 4.1: IBM's 8B Model Matching 32B MoE

firethering.com

181–190 of 223 posts

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#181
post #143

Earlier quoted context omitted.

Fellow kakoune user here. I'm curious about your use case/ what you're doing with it!

I'm just messing around with building agents, that's all. I'm not super interested in making ones that just sit in a terminal executing shell scripts because truth be told they're absolutely trivial to make and don't show any interesting parts of LLMs, whereas telling an agent that they are sitting in Kakoune is a whole lot more interesting and really shows a lot of what LLMs aren't great at, and how they'll have to…

As an addendum to this:

If I were to try to make something more useful out of this, I'd probably add the ability for LLMs to list buffers, probably give them an easier out for executing shell scripts in the way they prefer, give them an easier time to list docs and a few other things like that.

The tools and the interaction with Kakoune is really trivial to write; I already use this by having the agent write to the session FIFO (a very simple binary format) and I extract information via my own FIFO that Kakoune writes to (this is used for the buffer data only right now).

I think once you started using it more as a tool and not a pseudo-benchmark like I am you'd probably think of even more things to add but a lot of it comes down to just making Kakoune's state visible and making shell spam (which the LLMs love) easier.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#183
Very much an aside, but I'm struck by IBM's consistent iconic design language. For me it harkens all the way back to the futuristic design in 2001: A Space Odyssey from 1968. But you can also see it in their old mainframe hardware designs and other places.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#184

On the topic of local models, is there a good equivalent to something like Claude's chat interface? I've recently started transitioning to open models after getting fed up with Claude's usage limits (I'm not in a position to drop $200/month), and for coding tasks Kimi 2.6 has been about the same as Sonnet in my experience. The only thing I've found myself missing is a nice interface to ask it questions and have it he…

I re-created Claude's interface closely here, feel free to fork https://github.com/mudkipdev/chat

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#185

I test drove it yesterday. It's pretty impressive at 8b. Runs on commodity hardware quickly. Qwen3.6 35b a3b is still my local champion but I may use this for auto complete and small tasks. Granite has recent training data which is nice. If the other small models got fine tuned on recent data I don't know if I would use this at all, but that alone makes it pretty decent. The 4b they released was not good for my needs…

> Qwen3.6 35b a3b is still my local champion but I may use this for auto complete and small tasks. I second this! Using the Unsloth Q6 (I forgot the exact name). Currently using it with forgecode (with zsh), on my Strix Halo, and it's suprisingly really good. I would say slightly Similar to Haiku 4.5, plus additional privacy, minus speed. It's surprisingly really fast for the hardware, given the speculative decoding,…

If you use it for agentic coding and often hit PP, there's something wrong with your harness IMO

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#186

On the topic of local models, is there a good equivalent to something like Claude's chat interface? I've recently started transitioning to open models after getting fed up with Claude's usage limits (I'm not in a position to drop $200/month), and for coding tasks Kimi 2.6 has been about the same as Sonnet in my experience. The only thing I've found myself missing is a nice interface to ask it questions and have it he…

Yes but not exactly. - A lot of people suggesting llama-server's web ui, but that requires you use local AI (llama.cpp), it's persisting content into your browser rather than the server (so you can lose your chats), and it doesn't support much functionality. - There are some pure-browser chat interfaces that are like llama-server but you can use remote LLMs. This is closer to what you want, but everything is stored i…

I started using https://github.com/milisp/codexia/ (which is a desktop app or a web server) that wraps your regular codex-cli or Claude Code CLI. So you can see Codex/Claude threads in your web UI and access it remotely. I love it because you can do Web UI or terminal and all conversations are preserved.

Unfortunately it is pretty buggy, so I am maintaining a fork matching my personal needs with bugfixes and a few extra features.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#187

On the topic of local models, is there a good equivalent to something like Claude's chat interface? I've recently started transitioning to open models after getting fed up with Claude's usage limits (I'm not in a position to drop $200/month), and for coding tasks Kimi 2.6 has been about the same as Sonnet in my experience. The only thing I've found myself missing is a nice interface to ask it questions and have it he…

Codex cli is open source

v0.125.0 finally broke open models including their own gpt-oss over llama.cpp or vllm. I don't think they will fix it.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#188

Earlier quoted context omitted.

I'm pretty sure there's someone somewhere who'll create a proper harness that's equivalent to one giant model. The difficulty is mostly local hardware has lot of memory constraints. Targeting 128GB would seem to be the current sweet spot. If we could get out of the corporate market movers of buying up all the memory, we could maybe have more. Regardless, the people in the 80s capable of pruning programs to fit on sma…

What needs to happen is for companies (or individuals) tired of that to pool money together to build new, memory products. Then, sell them to consumers first and for non-AI use. If not that, then round-robin scheduling of quantities so the units are spread around more. If costs are high, they might reserve a certain percentage for big business at market prices (or just under) to cover the chip's mask costs. After DDR…

Time for my daily "HBF is coming" comment.

The next step for models is to put the weights on flash, connected with a very wide interface to the accelerator. The first users will be datacenters, but it should trickle down to consumer hardware eventually. A single 512GB stack is expected to cost about $200, and provide 1.6TB/s of reads.

You still need some fast DRAM for the KV cache and for activations, but weights should be sitting on flash.

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#189
me: "Bits 5-4 indicate the video mode", how do you derive VIDEO_MASK_00b = 0x03C0 from this?

Granite 4.1 8B: "I apologize for the confusion earlier. Let's clarify the derivation of the VIDEO__MASK_00b based on the description "Bits 5-6 indicate the video mode" from the equipment_list section.

Given that bits 5 and 6 are used for video mode, we need to create a mask that isolates these two bits:

Bit 5 is the least significant bit of the group (2^5 = 32 decimal, 0b00100000 in 8-bit form). Bit 6 is the next bit up (2^6 = 64 decimal, 0b01000000 in 8-bit form). Combining these bits: ...

Therefore, the correct mask for VIDEO_MASK_00b should be 0x00C0"

Errors on top of errors when converting description into binary numbers. Its hopeless for basic task like parsing/generating headers :(

Re: Granite 4.1: IBM's 8B Model Matching 32B MoE

#190

Earlier quoted context omitted.

Apache 2.0 License. Did you not click the link to the project? They even list it in the article. > Apache 2.0 across the board, so commercial use is clean. Did you just stop when you saw open source and come post this here because you couldn't be bothered to... look at the project and see it's cleanly and clearly listed. Edit: Like. I get it. It's fine to question open source. But this isn't hidden. It's repeated and…

If I give you an amd64 elf binary under Apache2 license, is it open source?

Open source for ML is more like Allen Institute's Olmo models.

https://allenai.org/olmo

I'm just giving it as an example. I haven't looked at Granite's repos.

Post reply on HN