'Western Qwen': IBM Wows with Granite 4 LLM Launch and Hybrid Mamba/Transformer
1–10 of 26 posts
Re: 'Western Qwen': IBM Wows with Granite 4 LLM Launch and Hybrid Mamba/Transformer
#2IBM announcement post is more informative than venturebeat
IBM Granite 4.0: hyper-efficient, high performance hybrid models for enterprise
https://www.ibm.com/new/announcements/ibm-granite-4-0-hyper-...
Re: 'Western Qwen': IBM Wows with Granite 4 LLM Launch and Hybrid Mamba/Transformer
#3Tried out the Ollama version and it's insanely fast with really good results for 1.9GB size. Supposed to have a 1M context window, would be interested where the speed goes then.
No Mamba in the Ollama version though.
Re: 'Western Qwen': IBM Wows with Granite 4 LLM Launch and Hybrid Mamba/Transformer
#4IBM announcement post is more informative than venturebeat IBM Granite 4.0: hyper-efficient, high performance hybrid models for enterprise https://www.ibm.com/new/announcements/ibm-granite-4-0-hyper-...
ISO 42001 certified.
> ISO/IEC 42001 is an international standard that specifies requirements for establishing, implementing, maintaining, and continually improving an Artificial Intelligence Management System (AIMS) within organizations. It is designed for entities providing or utilizing AI-based products or services, ensuring responsible development and use of AI systems.
https://www.iso.org/standard/42001
If anyone has access to ISO standards, I'm really curious what the practical effects of that certification is. I.e. what things does Granite have that others don't, because they had to add/do it to fulfill the certification.
The committee was formed in 2017, chaired by an AI expert: https://www.iso.org/committee/6794475.html
Re: 'Western Qwen': IBM Wows with Granite 4 LLM Launch and Hybrid Mamba/Transformer
#5Every technical paper I've read that IBM publish at an ML conference has been P-hacked to hell. Stay away.
Re: 'Western Qwen': IBM Wows with Granite 4 LLM Launch and Hybrid Mamba/Transformer
#6Tried out the Ollama version and it's insanely fast with really good results for 1.9GB size. Supposed to have a 1M context window, would be interested where the speed goes then. No Mamba in the Ollama version though.
(I've only just starting running local LLMs so excuse the dumb question).
Would Granite run with llama.cpp and use Mamba?
Re: 'Western Qwen': IBM Wows with Granite 4 LLM Launch and Hybrid Mamba/Transformer
#7Tried out the Ollama version and it's insanely fast with really good results for 1.9GB size. Supposed to have a 1M context window, would be interested where the speed goes then. No Mamba in the Ollama version though.
(I've only just starting running local LLMs so excuse the dumb question). Would Granite run with llama.cpp and use Mamba?
Last I checked Ollama inference is based on llama.cpp so either Ollama has not caught up yet, or the answer is no.
EDIT: Looks like Granite 4 hybrid architecture support was added to llama.cpp back in May: https://github.com/ggml-org/llama.cpp/pull/13550
Re: 'Western Qwen': IBM Wows with Granite 4 LLM Launch and Hybrid Mamba/Transformer
#8Every technical paper I've read that IBM publish at an ML conference has been P-hacked to hell. Stay away.
Links? Maybe just paper titles?
Re: 'Western Qwen': IBM Wows with Granite 4 LLM Launch and Hybrid Mamba/Transformer
#9I made some dynamic GGUFs for the 32B MoE model! Try:
./llama.cpp/llama-cli -hf unsloth/granite-4.0-h-small-GGUF:UD-Q4_K_XL
Also a support agent finetuning notebook with granite 4: https://colab.research.google.com/github/unslothai/notebooks...
Re: 'Western Qwen': IBM Wows with Granite 4 LLM Launch and Hybrid Mamba/Transformer
#10I really just want to know how it compares to ChatGPT and Claude at various tasks, but there aren’t any graphs for that.