Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bamba: An open-source LLM that crosses a transformer with an SSM

research.ibm.com

Post reply on HN
Print view View original

41–50 of 73 posts

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#41

Post by actionfromafar »

cwt137 wrote: post #40
gryfft wrote: post #30

Earlier quoted context omitted.

A stopped clock is right twice a day, but a running clock set to the wrong time is always wrong.

Not always true! Your statement is only true when the running clock's speed is the same as time. Thus, regular time and the clock's time will never meet. If the clock is running faster than regular time, it will at point catch up to regular time and thus be correct for a split second. If the clock is slower than regular time, regular time will catch up to the clock and the clock will be right for a split second.

If we are being pedantic, running clocks never run exactly the same as time. So they'll be right (very) much more seldom than the stopped clock, which is right twice a day.

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#43

Post by dudeinhawaii »

adt wrote: post #24

https://lifearchitect.ai/models-table/ Love those GPQA scores hovering around 5% when chance (on 4-way multi-choice) would have got them 25%!

or.. A stopped clock is right twice a day; a mis-prompted LLM is wrong 19 times out of 20—but only because we handed it the wrong instruction sheet.

Procedural error in testing perhaps? I'm not familiar with the methodology for GPQA.

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#45

Post by boroboro4 »

parrit wrote: post #35
quantadev wrote: post #21

Earlier quoted context omitted.

Not be contrarian, but if the next word prediction happens to be someone's name or a place or something discussed multiple places in the book then often, yes, a knowledge of the full plot of the book is "required" just to predict the next word, as you get to the middle or end of a book. For example you could never fill in the last chapter of any good book without having knowledge of every previous chapter. Not highly…

What an LLM does is stuff it all into short term memory. Humans dump the first pages into long term memory and "make sense" of it. Humans have a massive context window because of this (and sheer brain size and efficiency).

We don’t put things into long term memory after we read it. We usually put it after night of sleep. I personally think that context (and kv cache correspondingly) in the models are akin to our short term memory, while training process (and actual weights) are to our long term memory. And we can’t be sure our short term memory doesn’t work in a way of matching the current context towards currently stored short term memory. From this perspective transformers are enough and just fine.

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#46

Post by parrit »

boroboro4 wrote: post #45
parrit wrote: post #35

Earlier quoted context omitted.

What an LLM does is stuff it all into short term memory. Humans dump the first pages into long term memory and "make sense" of it. Humans have a massive context window because of this (and sheer brain size and efficiency).

We don’t put things into long term memory after we read it. We usually put it after night of sleep. I personally think that context (and kv cache correspondingly) in the models are akin to our short term memory, while training process (and actual weights) are to our long term memory. And we can’t be sure our short term memory doesn’t work in a way of matching the current context towards currently stored short term me…

So if you now hide my original comment and try to recall what I said, do you know it word for word (and are thinking if every word, e.g. did I use one or 2 spaces somewhere as that would change tokens) or do you have a rough concept of what I said?

OTOH if you had to remember a phone number to write it down, how does that differ?

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#47

Post by nathan_douglas »

cwt137 wrote: post #40
gryfft wrote: post #30

Earlier quoted context omitted.

A stopped clock is right twice a day, but a running clock set to the wrong time is always wrong.

Not always true! Your statement is only true when the running clock's speed is the same as time. Thus, regular time and the clock's time will never meet. If the clock is running faster than regular time, it will at point catch up to regular time and thus be correct for a split second. If the clock is slower than regular time, regular time will catch up to the clock and the clock will be right for a split second.

If the clock is running backwards at very high speed, it would be right infinitely many times but the proportion of the time that it is right would approach some finite constant.

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#49

Post by boroboro4 »

parrit wrote: post #46
boroboro4 wrote: post #45

Earlier quoted context omitted.

We don’t put things into long term memory after we read it. We usually put it after night of sleep. I personally think that context (and kv cache correspondingly) in the models are akin to our short term memory, while training process (and actual weights) are to our long term memory. And we can’t be sure our short term memory doesn’t work in a way of matching the current context towards currently stored short term me…

So if you now hide my original comment and try to recall what I said, do you know it word for word (and are thinking if every word, e.g. did I use one or 2 spaces somewhere as that would change tokens) or do you have a rough concept of what I said? OTOH if you had to remember a phone number to write it down, how does that differ?

I think in a way it makes transformers superior to humans, their short term memory is much more powerful =) Supporting extra long contexts also make transformers super human. Because, again, human's short term memory is exactly this - short term. And much shorter than millions of tokens we expect from models nowadays.

As for SSMs - I think they compress model memory state way too much. Mixed global/local attention layers do just as well. And sparse/block attention seems like a way forward much more (https://arxiv.org/abs/2502.11089).

Re: Bamba: An open-source LLM that crosses a transformer with an SSM

#50

Post by visarga »

bushbaba wrote: post #48

Wonder if the name is inspired by my favorite snack, bamba. The best are the hazelnut bamba. Btw bamba if given to kids at a young age can drastically reduce the chance of peanut allergies

Let me show you the etymology of Bamba:

SSM (state space model) -> SSSM (structured state space model) -> (it's like a snake ssss...) Mamba -> Bamba

Post reply on HN