Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bamba: An open-source LLM that crosses a transformer with an SSM

research.ibm.com

Post reply on HN
Print view View original

51–60 of 73 posts

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#51

Post by famouswaffles »

jmward01 wrote: post #13

This type of architecture is definitely the future. Unlimited attn is a dead end. As a human you don't need to scan an entire book just to guess what the next word will be and LLMs shouldn't need that either.

Humans can re-attend to material whenever necessary (i.e you can just re-read a book, re-watch a documentary etc when you feel you have missed crucial context) so it's not the end of the world. These SSMs or modern RNNs can't and if crucial context has been discarded by the end of the query then well too bad. Transformers are of course always re-attending so not an issue for them either. Until that issue is resolved, i don't think attention will be going anywhere.

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#52

Post by [deleted] »

boroboro4 wrote: post #45
parrit wrote: post #35

Earlier quoted context omitted.

What an LLM does is stuff it all into short term memory. Humans dump the first pages into long term memory and "make sense" of it. Humans have a massive context window because of this (and sheer brain size and efficiency).

We don’t put things into long term memory after we read it. We usually put it after night of sleep. I personally think that context (and kv cache correspondingly) in the models are akin to our short term memory, while training process (and actual weights) are to our long term memory. And we can’t be sure our short term memory doesn’t work in a way of matching the current context towards currently stored short term me…

[deleted]

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#53

Post by itayd »

kridsdale1 wrote: post #18
iddan wrote: post #16

Earlier quoted context omitted.

And in Heberw it's the name of a snack made of peanut-butter-flavored puffed maize https://en.wikipedia.org/wiki/Bamba_(snack)

I imported these to America to feed my infant. Data shows the prevalence of peanut allergies lines up with when AAP guidelines started recommending that babies do NOT eat peanut. Israel never went along with this and thus has the lowest rates of allergies in the world.

You actually don't need to self import these. Usually Safeway (is it only a west coast thing?) always have these stocked in the Kosher section.

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#54

Post by nthingtohide »

gryfft wrote: post #30
adt wrote: post #24

https://lifearchitect.ai/models-table/ Love those GPQA scores hovering around 5% when chance (on 4-way multi-choice) would have got them 25%!

A stopped clock is right twice a day, but a running clock set to the wrong time is always wrong.

> a running clock set to the wrong time is always wrong.

Could be right within 15 min accuracy in the appropriate timezone. And such a mechanism can be corrected for in the postprocessing step.

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#55

Post by zaptrem »

visarga wrote: post #50
bushbaba wrote: post #48

Wonder if the name is inspired by my favorite snack, bamba. The best are the hazelnut bamba. Btw bamba if given to kids at a young age can drastically reduce the chance of peanut allergies

Let me show you the etymology of Bamba: SSM (state space model) -> SSSM (structured state space model) -> (it's like a snake ssss...) Mamba -> Bamba

Where does the B come from?

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#56

Post by k__ »

cwt137 wrote: post #40
gryfft wrote: post #30

Earlier quoted context omitted.

A stopped clock is right twice a day, but a running clock set to the wrong time is always wrong.

Not always true! Your statement is only true when the running clock's speed is the same as time. Thus, regular time and the clock's time will never meet. If the clock is running faster than regular time, it will at point catch up to regular time and thus be correct for a split second. If the clock is slower than regular time, regular time will catch up to the clock and the clock will be right for a split second.

My girlfriend's microwave-clock runs faster than normal.

Somehow this thing manages to accumulate an error of ~15 minutes in a month.

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#57

Post by littlestymaar »

boroboro4 wrote: post #49
parrit wrote: post #46

Earlier quoted context omitted.

So if you now hide my original comment and try to recall what I said, do you know it word for word (and are thinking if every word, e.g. did I use one or 2 spaces somewhere as that would change tokens) or do you have a rough concept of what I said? OTOH if you had to remember a phone number to write it down, how does that differ?

I think in a way it makes transformers superior to humans, their short term memory is much more powerful =) Supporting extra long contexts also make transformers super human. Because, again, human's short term memory is exactly this - short term. And much shorter than millions of tokens we expect from models nowadays. As for SSMs - I think they compress model memory state way too much. Mixed global/local attention la…

> And much shorter than millions of tokens we expect from models nowadays.

Yet all current model still suck above 32k. (Yes some can do needle in a haystack fine, but they still fail at anything even slightly more complex over a long context).

32k is still much higher than humans' though, so I agree with you that it gives them some kind of super human abilities over moderately long context, but they are still disappointingly bad over longer context.

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#58

Post by patapong »

cwt137 wrote: post #40
gryfft wrote: post #30

Earlier quoted context omitted.

A stopped clock is right twice a day, but a running clock set to the wrong time is always wrong.

Not always true! Your statement is only true when the running clock's speed is the same as time. Thus, regular time and the clock's time will never meet. If the clock is running faster than regular time, it will at point catch up to regular time and thus be correct for a split second. If the clock is slower than regular time, regular time will catch up to the clock and the clock will be right for a split second.

And we haven't even touched on the issue of 24-hour format digital clocks, which can at most be right once per day if stopped!

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#59

Post by flaviolivolsi »

bushbaba wrote: post #48

Wonder if the name is inspired by my favorite snack, bamba. The best are the hazelnut bamba. Btw bamba if given to kids at a young age can drastically reduce the chance of peanut allergies

Bamba means cocaine in Italian. Better not to give it to kids

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#60

Post by anentropic »

> they added another trillion tokens and shrank the model from 18 GB to 9 GB through quantization, reducing its bit width from Mamba2’s 16-bit floating-point precision to 8-bits.

This sounds like what they call "Bamba-9B" is actually an 18B model quantised to 8 bits.

I thought generally we were naming models "nB" by their number of params and treating quantisation as a separate concern. Are there any other models that instead treat the name as an indicative memory requirement?

Is this an attempt to hide that it fares poorly vs other ~18B parameter models?

EDIT: no, I just misunderstood

Post reply on HN