Live data from Hacker News

Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

research.meta.ai

541–550 of 682 posts

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#541

https://xcancel.com/finkd/status/2086755195535413696 "... Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model..." This is bigger news - good for self hosting enthusiasts and a strategically sound move for Meta. Any push towards 'anti Chinese' models will directly benefit Meta as the competition on the frontier open-weights American models is almost non-existent. Meta will have no probl…

It wouldn’t surprise me if Meta does become the #1 American open weights provider, but I doubt it’ll be easy. Thinking Machines has a good amount of talent behind them as I understand it and their Inkling model was decent (admittedly not great though). I think Meta’s biggest problem is going to be internal as there’s be a bunch of headlines posted here on their talent retention issues.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#542

Remember when we needed 200 servers for an enterprise website because Apache used one process or thread per connection - and Nginx collapsed that into a single box overnight? That moment for LLMs is near. It’s going to move us from the big iron era of AI to small portable brains. Nature has already proved it’s possible with 20 watts and very little heat generation. And I think the data center buildout will end in car…

I don’t remember that and I was there! The idea that the performance delta between Apache and nginx for any normal workload is anything like 20,000% is absurd.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#543

> Muse Glimmer is a 30-billion-parameter model optimized for always-on local agent workflows. It’s small enough to run on a Mac or PC with a single consumer GPU, enabling use cases that range from local agents and function calling, to local coding, and LLM-as-a-judge evaluation. The next iteration in LLM products is a 24/7 thinking loop where the claude-code like thing gets input continuously from your wearable, noti…

This is is already possible with Claude Code. I use a setup where I have one instance monitoring a local queue, I have a web app for receiving webhooks from various sources and pushing them to the queue. Plus email for things that don't have webhooks. That instance then decides what to do with each input, sometimes it can spawn additional agent to investigate/prepare, sometimes it creates a ticket assigned to me and then waits for me input. All of that just uses the monitoring tools built into CC. The dispatcher loop doesn't need to be extremely smart, so I might experiment replacing it with a local model like this.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#544
post #423

Earlier quoted context omitted.

It's not perfect but it is very terse! Better than BottleCap managed to do with post-training Qwen in ThinkingCap. I suspect it will help a lot with enabling preserve-reasoning, because the biggest apparent limitation of this model is the 128K context window. Though the practical issue I am seeing on my M1 Max MBP is that performance suddenly drops off a cliff if I have DFlash enabled.

128k context window is a complete non-started for us. We need to optimize our most needy agentic jobs, but our average context is well above that

I was able to use rope/yarn scaling with llama.cpp to extend the context window to 256K and it seems to be pretty usable on the debugging-and-bugfixing session I have that’s 216K tokens deep. No infinite output loops, reasoning is still coherent, tool calls appear to be passing and failing at roughly the same rate as a fresh context window. Haven’t tried going to 512K or higher yet but 256K definitely seems usable to me on a single Radeon R9700.

I added these arguments to my llama-server call, using the official GGUF release from Meta’s account on HF.

`--rope-scaling yarn --rope-scale 2 --yarn-orig-ctx 131072 --override-kv muse-glimmer.context_length=int:262144,dflash.context_length=int:262144`

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#545

Optimizing speed is really the way to go. Yet 24GB is not what everyone can afford. Maybe we could take some of those 56tk/s and transfer into some free RAM space using MoE loading ? I'd be glad with a less than 10GB and more than 6tk/s model.

Unfortunately this is just the entry price for LLMs. With the exception of the Qwen 27B models, I personally haven’t found a ton of use cases for models less than 200B. With the right setup, fine tuning, etc, you can make small models do cool things, but hard to please everyone given the insane hardware costs at the moment and the comparably cheap API costs.

some small models are fast, and fine tuning can be done locally

for example in gaming context, if you need an answer below 5 seconds, they are the sweet spot

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#546

> Muse Glimmer is a 30-billion-parameter model optimized for always-on local agent workflows. It’s small enough to run on a Mac or PC with a single consumer GPU, enabling use cases that range from local agents and function calling, to local coding, and LLM-as-a-judge evaluation. The next iteration in LLM products is a 24/7 thinking loop where the claude-code like thing gets input continuously from your wearable, noti…

This is is already possible with Claude Code. I use a setup where I have one instance monitoring a local queue, I have a web app for receiving webhooks from various sources and pushing them to the queue. Plus email for things that don't have webhooks. That instance then decides what to do with each input, sometimes it can spawn additional agent to investigate/prepare, sometimes it creates a ticket assigned to me and…

If you are willing and not too busy, What model do you use and what is your cost? (If using subscription would you be able to check with 'npx ccusage').

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#547
post #522

Remember when we needed 200 servers for an enterprise website because Apache used one process or thread per connection - and Nginx collapsed that into a single box overnight? That moment for LLMs is near. It’s going to move us from the big iron era of AI to small portable brains. Nature has already proved it’s possible with 20 watts and very little heat generation. And I think the data center buildout will end in car…

Nature takes its own sweet time to come up with photosynthesis or the krebs cycle. What takes 2 billion years for Nature to work out, these large systems will soon do it in 2. They have capacity to compress time in ways the chimp troupe cant.

This is a statement of nearly pure faith not fact. Which is fine. I have a lot of things I believe based in pure faith. The difference is that I don't state them as if they were fact. Which you appear to be doing here.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#548
post #544

Earlier quoted context omitted.

128k context window is a complete non-started for us. We need to optimize our most needy agentic jobs, but our average context is well above that

I was able to use rope/yarn scaling with llama.cpp to extend the context window to 256K and it seems to be pretty usable on the debugging-and-bugfixing session I have that’s 216K tokens deep. No infinite output loops, reasoning is still coherent, tool calls appear to be passing and failing at roughly the same rate as a fresh context window. Haven’t tried going to 512K or higher yet but 256K definitely seems usable to…

Useful stuff, thank you. And more reading for me to do!

(DFlash seems rather weird on my M1 Max; model performance suddenly drops off a cliff like maybe there is some memory management issue).

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#549
post #345
post #327

The gguf is up and works, I don’t know if it’s them or unsloth that’s facilitated this but it’s nice because e.g. Inkling still doesn’t appear to have support in llama.cpp which makes it irrelevant to a class of user. Unfortunately I don’t have enough experience with Qwen 27B to immediately compare, but I do it’s Qwen 3.6 35B A3. It’s much slower obviously but it seems to be way more efficient with its thinking to th…

I don't really use the Qwen 3.6 27B though I do test the variants (Bonsai, ThinkingCap). I really like the 3.6 35B A3B for experiments, and it seems OK, but as you say, it spins round in thinking loops more than say the 26B Gemma 4 does. If Muse doesn't actually-wait itself as much it will be very interesting. I am just downloading it to run my small tests.

I have a custom A3B proxy that caps its thinking off. It is a known issue with the model that Qwen themselves documented but is almost never addressed in any harnesses. I also patched up a few other known bugs in the proxy. I highly recommend you shim A3B and when it hits 2K thinking tokens inject (paraphrasing) 'Time to wrap it up bud! Get to work' into its thinking stream. It almost always gets to work. If it needs more time to think there is always next turn.

In my experience it is almost never productively thinking past that point, just spinning in circles. I also reinject all of the thinking. And there are a few tells that it is getting stuck. I have an optional mode that takes the last few turns and tool calls and shoots it off to DSV4 with a prompt to basically understand where it is at and inject better thinking and or planning. It almost always gets it over relatively difficult humps, but some of the time I don't want things going remote. It might end up with 10-30 cents of DSV4 calls over a hours and the quality improvement is remarkable.

The other thing is I trick it into thinking a web_search tool is a web search but it really just asks DSV4 the prompt. DSV4 is a cheap filter to help prevent prompt injection lol. You can give it other models but DSV4 is my cheap-mode default.

edit: oh! My final 35B A3B tip -- use an extremely simple harness. Pi is good. Pi's default tools almost exactly match what Qwen says they tested the model with (likely meaning that tool set is also what they trained it with or something similar). So, in my experience bigger harnesses don't have a noticeable improve ment on tasks.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#550

> Muse Glimmer is a 30-billion-parameter model optimized for always-on local agent workflows. It’s small enough to run on a Mac or PC with a single consumer GPU, enabling use cases that range from local agents and function calling, to local coding, and LLM-as-a-judge evaluation. The next iteration in LLM products is a 24/7 thinking loop where the claude-code like thing gets input continuously from your wearable, noti…

Maybe it's my lack of imagination, but what do you imagine you'd be doing where you'd want to keep a computer busy overnight?

It seems like the purpose of humans isn't to keep machines busy. When our phone or laptop is idle, it's fine if it sleeps. And when we do want something, we'd rather not wait.

(Also, this new model seems to be designed to keep latency down, which is useful for interactive tasks.)

Post reply on HN