Earlier quoted context omitted.
I’m using the new oQe quants in case that matters.
i see! would you mind sharing your model string?
[dead]
Bonsai 27B: A 27B-Class model that runs on a phone
271–278 of 278 posts
Re: Bonsai 27B: A 27B-Class model that runs on a phone
#272Earlier quoted context omitted.
Pasta should be made of whole durum wheat.. which is 12g/100g. What's this "good" you speak of? There's certainly worse (soft white/red flour with germ/bran removed), but where do we find the +3g? https://en.wikipedia.org/wiki/Pasta
From your own link: > Main ingredients: Durum wheat flour, water/ eggs May be going out on a limb here but I think it might be the eggs?
[deleted]
Re: Bonsai 27B: A 27B-Class model that runs on a phone
#273Got this running on my phone. Unfortunately like other small models it hallucinates quite easily. eg asked it what Signoz is. It reckoned it is a woocommerce/shopify competitor aimed at India market
Same for Talos. Response is 100% hallucinations
Re: Bonsai 27B: A 27B-Class model that runs on a phone
#274Excuse my likely stupid question, but has anybody had some success using Claude Code with frontier agents (or Junie or anything else) to invoke local LLMs for specific sub-tasks or wrapped as skills? In other words, is there a way to use expensive, frontier models as orchestrators that manage local models to do the specialised coding tasks?
[deleted]
Re: Bonsai 27B: A 27B-Class model that runs on a phone
#275Excuse my likely stupid question, but has anybody had some success using Claude Code with frontier agents (or Junie or anything else) to invoke local LLMs for specific sub-tasks or wrapped as skills? In other words, is there a way to use expensive, frontier models as orchestrators that manage local models to do the specialised coding tasks?
[dead]
Re: Bonsai 27B: A 27B-Class model that runs on a phone
#276Earlier quoted context omitted.
Good spaghetti has 15% protein, 15g per 100g. Even the lowest quality one has 12g/100g.
Pasta should be made of whole durum wheat.. which is 12g/100g. What's this "good" you speak of? There's certainly worse (soft white/red flour with germ/bran removed), but where do we find the +3g? https://en.wikipedia.org/wiki/Pasta
> Each of the hundreds of varieties of wheat cultivated in the United States falls under one of six classes depending on its planting time, harvest, hardiness, color and shape. The protein content of these wheat varieties ranges from 10 to 15 percent; durum wheat is the second highest. One hundred grams of durum wheat has 13.68 grams of protein; soft red winter wheat has 10.35 grams; soft white wheat contains 10.69 grams; hard white wheat has 11.31 grams; and 100 grams of hard red winter wheat has 12.61. The highest protein content occurs in hard red spring wheat, which contains 15.4 grams of protein per 100 grams.
https://www.weekand.com/healthy-living/article/protein-durum...
Re: Bonsai 27B: A 27B-Class model that runs on a phone
#277Excuse my likely stupid question, but has anybody had some success using Claude Code with frontier agents (or Junie or anything else) to invoke local LLMs for specific sub-tasks or wrapped as skills? In other words, is there a way to use expensive, frontier models as orchestrators that manage local models to do the specialised coding tasks?
[flagged]
Re: Bonsai 27B: A 27B-Class model that runs on a phone
#278Earlier quoted context omitted.
I can report that it's working in oMLX. I've been experimenting with the ternary one; it is quite an impressive model! I've been grilling it on some deep learning/computer vision stuff and it's aced everything so far. Responses are thorough, accurate, sophisticated. General knowledge outside of CS doesn't seem as robust, which I expected. Honestly, I don't think the examples in the blog post do it justice.
Nice, I tried it too with oMLX — agreed, it seems very capable for coding! Was slightly underwhelmed by performance though. I got about ~24 t/s on the ternary version on my M2 Max 64. That’s quite a bit slower than Qwen A3B 35B (4 bit unsloth). How was perf for you?
i'm getting similar toks on my 36gb m3 max. new omlx release (0.5.2) has dedicated decode kernels for the bonsai models — see if this improves perf for you.