Dear IBM name pickers: "Bamba", in Italian, means cocaine.
When I read the title 'IBM crossed a transformer with an SSM and got ‘Bamba’' I laughed so hard I woke up my kid
Bamba: An open-source LLM that crosses a transformer with an SSM
31–40 of 73 posts
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#32Another recent transformer/SSM hybrid is "M1", with a more than 3x claimed inference speed-up compared to equivalent transformers: https://arxiv.org/pdf/2504.10449
IBM is claiming at least a 2x inference speed-up with Bamba. Both groups say that future SSM optimizations to vLLM would lead to further inference speed improvement.
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#33https://lifearchitect.ai/models-table/ Love those GPQA scores hovering around 5% when chance (on 4-way multi-choice) would have got them 25%!
So could do better than chance by excluding the option it's picked?
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#34https://lifearchitect.ai/models-table/ Love those GPQA scores hovering around 5% when chance (on 4-way multi-choice) would have got them 25%!
A stopped clock is right twice a day, but a running clock set to the wrong time is always wrong.
The RMS of wrongness of the running clock is probably lower.
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#35This type of architecture is definitely the future. Unlimited attn is a dead end. As a human you don't need to scan an entire book just to guess what the next word will be and LLMs shouldn't need that either.
Not be contrarian, but if the next word prediction happens to be someone's name or a place or something discussed multiple places in the book then often, yes, a knowledge of the full plot of the book is "required" just to predict the next word, as you get to the middle or end of a book. For example you could never fill in the last chapter of any good book without having knowledge of every previous chapter. Not highly…
What an LLM does is stuff it all into short term memory. Humans dump the first pages into long term memory and "make sense" of it. Humans have a massive context window because of this (and sheer brain size and efficiency).
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#36Dear IBM name pickers: "Bamba", in Italian, means cocaine.
about time they did something to liven things up at big blue
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#37Dear IBM name pickers: "Bamba", in Italian, means cocaine.
A very funny and friendly way to say "cocaine" among italians.
I'm struggling to read it seriously.
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#38Dear IBM name pickers: "Bamba", in Italian, means cocaine.
Seems like a good fit.
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#39Never got how mamba models work in multiple dimensions and non-causally.
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#40https://lifearchitect.ai/models-table/ Love those GPQA scores hovering around 5% when chance (on 4-way multi-choice) would have got them 25%!
A stopped clock is right twice a day, but a running clock set to the wrong time is always wrong.
Not always true! Your statement is only true when the running clock's speed is the same as time. Thus, regular time and the clock's time will never meet.
If the clock is running faster than regular time, it will at point catch up to regular time and thus be correct for a split second. If the clock is slower than regular time, regular time will catch up to the clock and the clock will be right for a split second.