For some reason this link isn't loading, but it's on https://archive.ph/Ks0xt
Bamba: An open-source LLM that crosses a transformer with an SSM
11–20 of 73 posts
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#12Earlier quoted context omitted.
Maybe? https://en.m.wikipedia.org/wiki/Bamba_(snack) ;)
Or https://en.wikipedia.org/wiki/La_Bamba_(song)
Or (where I'm from) a school cafeteria:
https://www.thelocal.se/20221125/swedish-word-of-the-day-bam...
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#13This type of architecture is definitely the future. Unlimited attn is a dead end. As a human you don't need to scan an entire book just to guess what the next word will be and LLMs shouldn't need that either.
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#14Dear IBM name pickers: "Bamba", in Italian, means cocaine.
It's just a mamba (https://github.com/state-spaces/mamba) but with a transformer. Idk where the B comes from.
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#15Earlier quoted context omitted.
Maybe? https://en.m.wikipedia.org/wiki/Bamba_(snack) ;)
Or https://en.wikipedia.org/wiki/La_Bamba_(song)
Spot on. From the linked blog post "The refrain of La Bamba, the Mexican folk song that Ritchie Valens made famous, goes: Para bailar La Bamba/Se necesita una poca de Gracia. "
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#16Dear IBM name pickers: "Bamba", in Italian, means cocaine.
And in Heberw it's the name of a snack made of peanut-butter-flavored puffed maize
https://en.wikipedia.org/wiki/Bamba_(snack)
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#17Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#18Dear IBM name pickers: "Bamba", in Italian, means cocaine.
And in Heberw it's the name of a snack made of peanut-butter-flavored puffed maize https://en.wikipedia.org/wiki/Bamba_(snack)
I imported these to America to feed my infant. Data shows the prevalence of peanut allergies lines up with when AAP guidelines started recommending that babies do NOT eat peanut. Israel never went along with this and thus has the lowest rates of allergies in the world.
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#19Where's the code?
Re: Bamba: An open-source LLM that crosses a transformer with an SSM
#20LLM/state space models have been popular for some years now, see: https://arxiv.org/abs/2212.14052
More recently, hybrid architectures that utilize attention plus other operators are gaining traction.