Live data from Hacker News

Bonsai 27B: A 27B-Class model that runs on a phone

prismml.com

241–250 of 278 posts

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#241
post #235

Excuse my likely stupid question, but has anybody had some success using Claude Code with frontier agents (or Junie or anything else) to invoke local LLMs for specific sub-tasks or wrapped as skills? In other words, is there a way to use expensive, frontier models as orchestrators that manage local models to do the specialised coding tasks?

That is my entire workflow in opencode through delegates. I have an "orchestrator" which uses matt pocock skills like grilling and writes specs in gh tickets. Once that is done it delegates individual tickets to deepseek v4 flash based developer subagent which executes the code pretty fast. This agent can only delegate one subagent which is a code-reviewer which uses a better model like qwen 3.7 for review. So dev does its own review loop before returning.

It's going amazingly. The orchestrator holds the big knowledge from grilling and also enables me to do more grilling to refine the specs. The trick was to check and re-align after each phase was developed. Also carefully defining each workflow step explicitly otherwise it makes mistakes like trying to self review the code. Also needed to define very explicit contracts. I chose the same surfaces as the matt skill set.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#242
post #235

Excuse my likely stupid question, but has anybody had some success using Claude Code with frontier agents (or Junie or anything else) to invoke local LLMs for specific sub-tasks or wrapped as skills? In other words, is there a way to use expensive, frontier models as orchestrators that manage local models to do the specialised coding tasks?

why would you want to use claude code?

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#243
post #132

Earlier quoted context omitted.

I've got both that app and a 17 Pro, but it only lists one of the older Bonsai models not the 27B for me

I only have an iPhone 14 Pro, but under "manage models" it's showing Bonsai 8B and Ternary Bonsai 8B.

Ok figured it out - it’s linked to app version. Update Locally app and it shows up. No multimodal like the demo though

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#245

Earlier quoted context omitted.

Mixture of Experts is absolutely not what they're describing. MoE has to be one of the most misleadingly named things ever. It's completely confusing as to what it actually is.

MoE is literally exactly what they're describing. The classifier being described is baked into the model and is the thing that makes it MoE. Imagine I had [a model that was good at math], [model that was good at code], [model that was good at writing], [model that was good at general knowledge]. If I then had [a model that was good at determining whether the user query would be best served by one of those models and…

only there's no [a model that was good at math], [model that was good at code]... in the MoE

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#246
post #235

Excuse my likely stupid question, but has anybody had some success using Claude Code with frontier agents (or Junie or anything else) to invoke local LLMs for specific sub-tasks or wrapped as skills? In other words, is there a way to use expensive, frontier models as orchestrators that manage local models to do the specialised coding tasks?

This may not be exactly what you're asking, but I've been running Hermes using different frontier models (like DeepseekV4-Pro, GLM-5.2, and others) as the heavy lifters but who also spawn agents to run my local models (mainly Qwen3.6-IQ4) for offloading tasks that are a good fit for a quantized model that's local. Works really well.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#247

Earlier quoted context omitted.

Bitter lesson is knocking. Mixture of experts is essentially what you’re describing but free from unnecessary inductive biases.

Mixture of Experts is absolutely not what they're describing. MoE has to be one of the most misleadingly named things ever. It's completely confusing as to what it actually is.

Note the “but free from unnecessary inductive biases” part of my comment. By that I meant a decision to make each expert good at a human-defined thing.

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#248
post #235

Excuse my likely stupid question, but has anybody had some success using Claude Code with frontier agents (or Junie or anything else) to invoke local LLMs for specific sub-tasks or wrapped as skills? In other words, is there a way to use expensive, frontier models as orchestrators that manage local models to do the specialised coding tasks?

Look into FastContext by Microsoft. Not extraordinary but specifically designed for paired usage with a larger LLM to save tokens [1]. [1]: https://github.com/microsoft/fastcontext

This may be useful in this context: https://entelligentsia.github.io/is-grep-enough/fastcontext....

Re: Bonsai 27B: A 27B-Class model that runs on a phone

#249
post #235

Excuse my likely stupid question, but has anybody had some success using Claude Code with frontier agents (or Junie or anything else) to invoke local LLMs for specific sub-tasks or wrapped as skills? In other words, is there a way to use expensive, frontier models as orchestrators that manage local models to do the specialised coding tasks?

That is my entire workflow in opencode through delegates. I have an "orchestrator" which uses matt pocock skills like grilling and writes specs in gh tickets. Once that is done it delegates individual tickets to deepseek v4 flash based developer subagent which executes the code pretty fast. This agent can only delegate one subagent which is a code-reviewer which uses a better model like qwen 3.7 for review. So dev do…

Do you have source code or config for this setup ?
Post reply on HN