Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bonsai 27B: A 27B-Class model that runs on a phone

prismml.com

Post reply on HN
Print view View original

161–170 of 278 posts

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#161

Post by stamps »

gnabgib wrote: post #159
stamps wrote: post #153

Earlier quoted context omitted.

I know felcetti has a few varieties that exceed that number. ~112g for 16g protein.

The brand that uses khorasan wheat in some versions? Yeah they're a little higher (14.7g/100g). We're stretching the definition of pasta now (it's certainly the highest protein grain you can find)

Yeah I’m not 100% sure, they claim “special durum wheat variety.”

For the price, from a protein perspective, it would be cheaper to get it elsewhere.

https://shopfelicettipasta.com/products/monograno-felicetti-...

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#162

Post by ExxKA »

From an investors perspective, this is truly a paradigm shift - this will kill a whole range of startups in Europe which were packaging privacy and wrapping around large hosted models. There's absolutely no reason to use a "Privacy GPT tm" provider, then I have it all on my own laptop - There is also no need for banks or other regulated institutions to rely on those providers when they can selfhost with this much intelligence on tap.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#163

Post by tharkun__ »

janalsncm wrote: post #123
SwellJoe wrote: post #59

What I most want to see it compared to is Gemma 4 12B in the 4-bit QAT version. It's barely bigger than this at just under 7GB, so it also runs on just about any modern device and is remarkably smart for its size. It's an excellent tool user, crazy good vision for its size. I'm still trying to wrap my head around how much is lost with each step down in resolution, but the QAT versions from Google seem to prove the an…

Based on their numbers and cross referencing with the Gemma numbers, this model crushes Gemma 4 12b on math and coding, is slightly worse on knowledge and tool calling, and is significantly worse on vision tasks.

From my own experiments with local, low VRAM model use vs. what I'm used to from using Claude at work is that being good at "coding" is of no use, if you're worse at "tool calling" as coding in an agentic way requires quite a bit of tool calling.

If you can "hide" different models of 8GB VRAM requirements each that have those specialties and mix and match them for me without having to manage it manually, I'll be impressed. Until then I will keep using my Claude, because "remarkably good _for their size_" models I've tried so far just sucked at trying to use them the way I code at work with Claude.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#164

Post by brcmthrowaway »

mandeepj wrote: post #135
brcmthrowaway wrote: post #100

Earlier quoted context omitted.

I remember the infographics of 5-10 years ago. They had 200 billion. What happened? Sinking ship?

They spent close to $100 billion in stock buybacks during the last 10ish years :-)

How does that compare to other companies?

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#165

Post by gnabgib »

stamps wrote: post #161
gnabgib wrote: post #159

Earlier quoted context omitted.

The brand that uses khorasan wheat in some versions? Yeah they're a little higher (14.7g/100g). We're stretching the definition of pasta now (it's certainly the highest protein grain you can find)

Yeah I’m not 100% sure, they claim “special durum wheat variety.” For the price, from a protein perspective, it would be cheaper to get it elsewhere. https://shopfelicettipasta.com/products/monograno-felicetti-...

They do, but they have to use the standard numbers for durum (like everyone else, even growers - individual berries vary, of course). The only one I can find with these high numbers is the khorasan wheat versions[0].

[0]: https://m.media-amazon.com/images/I/81UbkYz+pZL._SL1500_.jpg

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#166

Post by mullingitover »

gnabgib wrote: post #150
purerandomness wrote: post #145

Earlier quoted context omitted.

Good spaghetti has 15% protein, 15g per 100g. Even the lowest quality one has 12g/100g.

Pasta should be made of whole durum wheat.. which is 12g/100g. What's this "good" you speak of? There's certainly worse (soft white/red flour with germ/bran removed), but where do we find the +3g? https://en.wikipedia.org/wiki/Pasta

From your own link:

> Main ingredients: Durum wheat flour, water/eggs

May be going out on a limb here but I think it might be the eggs?

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#167

Post by raylad »

raylad wrote: post #111

Not impressed. It fails the "Jabberwocky" test.

This got a downvote and I understand why: because I didn't describe the test, which is to ask it "Please recite Jabberwocky".

This is actually difficult because there are so many invented words in the poem which have extremely low frequencies in the training data. So a model that can do it properly is likely to be very good in other ways. Qwen-3.6-27B can do this until it gets overly quantized.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#168

Post by dofm »

ExxKA wrote: post #162

From an investors perspective, this is truly a paradigm shift - this will kill a whole range of startups in Europe which were packaging privacy and wrapping around large hosted models. There's absolutely no reason to use a "Privacy GPT tm" provider, then I have it all on my own laptop - There is also no need for banks or other regulated institutions to rely on those providers when they can selfhost with this much int…

It will when they can get the performance up a bit.

My brief experiments with the ternary version suggest that it broadly meets their claim to be a 27B model that fits in much less RAM, that is for sure. It is about as fast as the underlying Qwen 27B but it gets stuck in reasoning loops quite easily.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#170

Post by aesthesia »

gunalx wrote: post #32
luckystarr wrote: post #22

Tried it on Android and got "!!!!!!!!!!!!!" for answers.

The qwen models really seem to have this as a failure mode, its so annoying having a proper trace ending up in !!!!!! Garbage.

Oh, interesting. "!" is token id 0 in the Qwen tokenizer; I wonder if there's some tokenizer shenanigans either in inference or training that end up causing this specific behavior.
Post reply on HN