Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bonsai 27B: A 27B-Class model that runs on a phone

prismml.com

Post reply on HN
Print view View original

81–90 of 278 posts

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#81

Post by ndrwdvvs »

trollbridge wrote: post #27
sigbottle wrote: post #16

What's the hiring space and business strategy around all of these smaller AI labs? Its really cool that people like these guys get paid to optimize models and give them out for free (open source). Do a lot of these labs have forward deployed engineers doing integrations with customers who want local models? Is there a general shift towards the local model crowd?

If you read to the bottom of the page, it says they're funded by a few people, and one of them is Samsung. I'm betting Samsung wants to be able to ship a capable AI system on a future model of their phone so they can compete with Apple.

or smart fridge

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#82

Post by Havoc »

Catloafdev wrote: post #19
Havoc wrote: post #7

This must be some sort of unpublished app? I can just see their image tool on the app store

It's a LLM model, not a phone app. Available on HuggingFace: https://huggingface.co/collections/prism-ml/bonsai-27b

Indeed.

The article is about running it on a phone though, and shows an app with their branding running this in text mode on a phone. I'm asking where can I find this app to try what is being demonstrated in this article & video? Appstore only has an image gen app by them and other MLX apps I've tried don't seem to support this model

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#83

Post by thakoppno »

CharlesW wrote: post #58
kristianp wrote: post #21

Apparently Apple is "in talks" with the PrismML: https://www.cnbc.com/2026/07/14/apple-prismml-ai-compression...

Notably, PrismML CEO Babak Hassibi told CNBC this, so it’s either (1) bullshit, or (2) he just ended any chance of a relationship by leaking news of the talks.

either way, maybe a portent for the times.

apple’s secrecy agenda has been defeated to an extent by the practicalities of ubiquitous technology?

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#84

Post by verdverm »

drob518 wrote: post #74
verdverm wrote: post #36

Earlier quoted context omitted.

Orinth was not impressive in my vibes testing, I just completed my first grid analysis with real evals on qwen 27b. I can now scale that grid analysis and intend to include the qwen 9b ftunes I've seen going around. They were actually a main motivation because so many claim this or that one is better, but very little in the way of evals

I tried it, too, and it got stuck in some loops where it couldn’t recover. Shame, it was promising for the same reason as Bonsai’s models.

check out geyron-9b, I've only used it a bit, but seems better than orinth on vibe evals

huggingface.co/Tivaphraen/Geryon-9B-v1

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#85

Post by amelius »

gunalx wrote: post #32
luckystarr wrote: post #22

Tried it on Android and got "!!!!!!!!!!!!!" for answers.

The qwen models really seem to have this as a failure mode, its so annoying having a proper trace ending up in !!!!!! Garbage.

Wait in a regular sentence, what is the probability of "!!!" being followed by "!"?

Sounds like the model is not following a proper probabilistic choice here, so maybe more a programming error than a model training error.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#87

Post by segmondy »

rogerkirkness wrote: post #67
CharlesW wrote: post #58

Earlier quoted context omitted.

Notably, PrismML CEO Babak Hassibi told CNBC this, so it’s either (1) bullshit, or (2) he just ended any chance of a relationship by leaking news of the talks.

Apple would punish him severely unless they cleared it in advance, it might be to their advantage for some reason (negotiating with Google for Gemma rights? idk).

Apple is too desperate to be making demands. You confuse the Apple of yesterday and of today. Things change fast and things have changed.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#88

Post by drob518 »

verdverm wrote: post #84
drob518 wrote: post #74

Earlier quoted context omitted.

I tried it, too, and it got stuck in some loops where it couldn’t recover. Shame, it was promising for the same reason as Bonsai’s models.

check out geyron-9b, I've only used it a bit, but seems better than orinth on vibe evals huggingface.co/Tivaphraen/Geryon-9B-v1

Interesting, thanks. Looking at the model card on Huggingface, it’s combining the Qwythos and Qwable fine tunes from Empero.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#89

Post by kroaton »

Catloafdev wrote: post #18
xyzsparetimexyz wrote: post #10

That's awesome. What's the largest model that could fit onto a single 16gb gpu at 1.125 effects bits per weight?

Doing some naive math, the F16 filesize is ~53.8gb, the 1-bit version is ~3.8gb, about 7% of the original size. The F16 size is roughly 2x param count, so that gives a rough ballpark of ~110B.

Which would be very interesting to test, as larger models (such as Deepseek V4 Flash or Qwen 397B) seem to compress better. Their Q2 quants are usable as is, even without the ternary compression.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#90

Post by jldugger »

amelius wrote: post #85
gunalx wrote: post #32

Earlier quoted context omitted.

The qwen models really seem to have this as a failure mode, its so annoying having a proper trace ending up in !!!!!! Garbage.

Wait in a regular sentence, what is the probability of "!!!" being followed by "!"? Sounds like the model is not following a proper probabilistic choice here, so maybe more a programming error than a model training error.

After the third !, the probability of a fourth probably skyrockets =)
Post reply on HN