Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bonsai 27B: A 27B-Class model that runs on a phone

prismml.com

Post reply on HN
Print view View original

131–140 of 278 posts

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#131

Post by SwellJoe »

janalsncm wrote: post #123
SwellJoe wrote: post #59

What I most want to see it compared to is Gemma 4 12B in the 4-bit QAT version. It's barely bigger than this at just under 7GB, so it also runs on just about any modern device and is remarkably smart for its size. It's an excellent tool user, crazy good vision for its size. I'm still trying to wrap my head around how much is lost with each step down in resolution, but the QAT versions from Google seem to prove the an…

Based on their numbers and cross referencing with the Gemma numbers, this model crushes Gemma 4 12b on math and coding, is slightly worse on knowledge and tool calling, and is significantly worse on vision tasks.

To be fair, everything (roughly within an order of magnitude in size) is worse on vision. 12b is a beast for vision tasks, better than its bigger siblings, even.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#132

Post by Havoc »

smallerize wrote: post #93
Havoc wrote: post #7

This must be some sort of unpublished app? I can just see their image tool on the app store

One of the links on the sidebar goes to "Locally AI" https://apps.apple.com/us/app/locally-ai-by-lm-studio/id6741... it requires an iPhone 17 Pro or Pro Max to run the 27B model though.

I've got both that app and a 17 Pro, but it only lists one of the older Bonsai models not the 27B for me

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#133

Post by recursivegirth »

janalsncm wrote: post #123
SwellJoe wrote: post #59

What I most want to see it compared to is Gemma 4 12B in the 4-bit QAT version. It's barely bigger than this at just under 7GB, so it also runs on just about any modern device and is remarkably smart for its size. It's an excellent tool user, crazy good vision for its size. I'm still trying to wrap my head around how much is lost with each step down in resolution, but the QAT versions from Google seem to prove the an…

Based on their numbers and cross referencing with the Gemma numbers, this model crushes Gemma 4 12b on math and coding, is slightly worse on knowledge and tool calling, and is significantly worse on vision tasks.

I think this is where leveraging classifier models will become important. The frontier LLM models do "everything", while we've known for a while that to truly scale this we will need to distill models into their individual functions. I don't see this as necessarily a bad thing and hope more is done in this space. Very promising.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#134

Post by smallerize »

Havoc wrote: post #132
smallerize wrote: post #93

Earlier quoted context omitted.

One of the links on the sidebar goes to "Locally AI" https://apps.apple.com/us/app/locally-ai-by-lm-studio/id6741... it requires an iPhone 17 Pro or Pro Max to run the 27B model though.

I've got both that app and a 17 Pro, but it only lists one of the older Bonsai models not the 27B for me

I only have an iPhone 14 Pro, but under "manage models" it's showing Bonsai 8B and Ternary Bonsai 8B.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#135

Post by mandeepj »

brcmthrowaway wrote: post #100
conception wrote: post #97

Earlier quoted context omitted.

You’re right, Apple only has 68 billion dollars in cash, up 40% since last year. Definitely on their last legs.

I remember the infographics of 5-10 years ago. They had 200 billion. What happened? Sinking ship?

They spent close to $100 billion in stock buybacks during the last 10ish years :-)

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#136

Post by recursivegirth »

sandworm101 wrote: post #120
RugnirViking wrote: post #106

maybe its nitpicking here but the demo shows them asking the model what to cook and its recipie sounds like it wouldn't be very good and also it totally gets the macronutrients wrong. 25g protein for "spaghetti, carrots, peppers, garlic and herbs"?

And why would i want such mundane questions to be handled by an AI on my phone? That sort of thing doesnt need AI, let alone a local one. Basic google search was answering those questions long ago. My point: phone-sized AI is only useful if it can do things that only AI can do. Can it ingest a document scanned by the phones camera? Can it translate in real time? I dont see how or why i would ever ask it for recipe ad…

Awwwe, cmon. You are thinking about the problem space incorrectly. This is an opportunity to create a unicorn company that develops the first AI tongue.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#137

Post by bromuro »

RugnirViking wrote: post #106

maybe its nitpicking here but the demo shows them asking the model what to cook and its recipie sounds like it wouldn't be very good and also it totally gets the macronutrients wrong. 25g protein for "spaghetti, carrots, peppers, garlic and herbs"?

200gr spaghetti do actually contain ~25g protein

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#138

Post by codebje »

sandworm101 wrote: post #120
RugnirViking wrote: post #106

maybe its nitpicking here but the demo shows them asking the model what to cook and its recipie sounds like it wouldn't be very good and also it totally gets the macronutrients wrong. 25g protein for "spaghetti, carrots, peppers, garlic and herbs"?

And why would i want such mundane questions to be handled by an AI on my phone? That sort of thing doesnt need AI, let alone a local one. Basic google search was answering those questions long ago. My point: phone-sized AI is only useful if it can do things that only AI can do. Can it ingest a document scanned by the phones camera? Can it translate in real time? I dont see how or why i would ever ask it for recipe ad…

If it can give me the recipe without 14 pages of backstory about how Nonna used to make it, it'd be satisfying a real need.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#140

Post by SamBam »

nozzlegear wrote: post #117
RugnirViking wrote: post #106

maybe its nitpicking here but the demo shows them asking the model what to cook and its recipie sounds like it wouldn't be very good and also it totally gets the macronutrients wrong. 25g protein for "spaghetti, carrots, peppers, garlic and herbs"?

I personally don't like carrots much, but it doesn't sound bad to me – could definitely use a tomato sauce but that doesn't seem to be an option from the image it was given. > 25g protein for "spaghetti, carrots, peppers, garlic and herbs"? Maybe it assumed the pasta was some kind of protein chickpea pasta? =P it definitely seems wrong.

Many people don't realize that regular spaghetti actually is pretty high in protein. All that gluten.
Post reply on HN