https://xcancel.com/finkd/status/2086755195535413696 "... Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model..." This is bigger news - good for self hosting enthusiasts and a strategically sound move for Meta. Any push towards 'anti Chinese' models will directly benefit Meta as the competition on the frontier open-weights American models is almost non-existent. Meta will have no probl…
Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
541–550 of 682 posts
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#542Remember when we needed 200 servers for an enterprise website because Apache used one process or thread per connection - and Nginx collapsed that into a single box overnight? That moment for LLMs is near. It’s going to move us from the big iron era of AI to small portable brains. Nature has already proved it’s possible with 20 watts and very little heat generation. And I think the data center buildout will end in car…
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#543> Muse Glimmer is a 30-billion-parameter model optimized for always-on local agent workflows. It’s small enough to run on a Mac or PC with a single consumer GPU, enabling use cases that range from local agents and function calling, to local coding, and LLM-as-a-judge evaluation. The next iteration in LLM products is a 24/7 thinking loop where the claude-code like thing gets input continuously from your wearable, noti…
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#544Earlier quoted context omitted.
It's not perfect but it is very terse! Better than BottleCap managed to do with post-training Qwen in ThinkingCap. I suspect it will help a lot with enabling preserve-reasoning, because the biggest apparent limitation of this model is the 128K context window. Though the practical issue I am seeing on my M1 Max MBP is that performance suddenly drops off a cliff if I have DFlash enabled.
128k context window is a complete non-started for us. We need to optimize our most needy agentic jobs, but our average context is well above that
I added these arguments to my llama-server call, using the official GGUF release from Meta’s account on HF.
`--rope-scaling yarn --rope-scale 2 --yarn-orig-ctx 131072 --override-kv muse-glimmer.context_length=int:262144,dflash.context_length=int:262144`
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#545Optimizing speed is really the way to go. Yet 24GB is not what everyone can afford. Maybe we could take some of those 56tk/s and transfer into some free RAM space using MoE loading ? I'd be glad with a less than 10GB and more than 6tk/s model.
Unfortunately this is just the entry price for LLMs. With the exception of the Qwen 27B models, I personally haven’t found a ton of use cases for models less than 200B. With the right setup, fine tuning, etc, you can make small models do cool things, but hard to please everyone given the insane hardware costs at the moment and the comparably cheap API costs.
for example in gaming context, if you need an answer below 5 seconds, they are the sweet spot
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#546> Muse Glimmer is a 30-billion-parameter model optimized for always-on local agent workflows. It’s small enough to run on a Mac or PC with a single consumer GPU, enabling use cases that range from local agents and function calling, to local coding, and LLM-as-a-judge evaluation. The next iteration in LLM products is a 24/7 thinking loop where the claude-code like thing gets input continuously from your wearable, noti…
This is is already possible with Claude Code. I use a setup where I have one instance monitoring a local queue, I have a web app for receiving webhooks from various sources and pushing them to the queue. Plus email for things that don't have webhooks. That instance then decides what to do with each input, sometimes it can spawn additional agent to investigate/prepare, sometimes it creates a ticket assigned to me and…
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#547Remember when we needed 200 servers for an enterprise website because Apache used one process or thread per connection - and Nginx collapsed that into a single box overnight? That moment for LLMs is near. It’s going to move us from the big iron era of AI to small portable brains. Nature has already proved it’s possible with 20 watts and very little heat generation. And I think the data center buildout will end in car…
Nature takes its own sweet time to come up with photosynthesis or the krebs cycle. What takes 2 billion years for Nature to work out, these large systems will soon do it in 2. They have capacity to compress time in ways the chimp troupe cant.
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#548Earlier quoted context omitted.
128k context window is a complete non-started for us. We need to optimize our most needy agentic jobs, but our average context is well above that
I was able to use rope/yarn scaling with llama.cpp to extend the context window to 256K and it seems to be pretty usable on the debugging-and-bugfixing session I have that’s 216K tokens deep. No infinite output loops, reasoning is still coherent, tool calls appear to be passing and failing at roughly the same rate as a fresh context window. Haven’t tried going to 512K or higher yet but 256K definitely seems usable to…
(DFlash seems rather weird on my M1 Max; model performance suddenly drops off a cliff like maybe there is some memory management issue).
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#549The gguf is up and works, I don’t know if it’s them or unsloth that’s facilitated this but it’s nice because e.g. Inkling still doesn’t appear to have support in llama.cpp which makes it irrelevant to a class of user. Unfortunately I don’t have enough experience with Qwen 27B to immediately compare, but I do it’s Qwen 3.6 35B A3. It’s much slower obviously but it seems to be way more efficient with its thinking to th…
I don't really use the Qwen 3.6 27B though I do test the variants (Bonsai, ThinkingCap). I really like the 3.6 35B A3B for experiments, and it seems OK, but as you say, it spins round in thinking loops more than say the 26B Gemma 4 does. If Muse doesn't actually-wait itself as much it will be very interesting. I am just downloading it to run my small tests.
In my experience it is almost never productively thinking past that point, just spinning in circles. I also reinject all of the thinking. And there are a few tells that it is getting stuck. I have an optional mode that takes the last few turns and tool calls and shoots it off to DSV4 with a prompt to basically understand where it is at and inject better thinking and or planning. It almost always gets it over relatively difficult humps, but some of the time I don't want things going remote. It might end up with 10-30 cents of DSV4 calls over a hours and the quality improvement is remarkable.
The other thing is I trick it into thinking a web_search tool is a web search but it really just asks DSV4 the prompt. DSV4 is a cheap filter to help prevent prompt injection lol. You can give it other models but DSV4 is my cheap-mode default.
edit: oh! My final 35B A3B tip -- use an extremely simple harness. Pi is good. Pi's default tools almost exactly match what Qwen says they tested the model with (likely meaning that tool set is also what they trained it with or something similar). So, in my experience bigger harnesses don't have a noticeable improve ment on tasks.
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#550> Muse Glimmer is a 30-billion-parameter model optimized for always-on local agent workflows. It’s small enough to run on a Mac or PC with a single consumer GPU, enabling use cases that range from local agents and function calling, to local coding, and LLM-as-a-judge evaluation. The next iteration in LLM products is a 24/7 thinking loop where the claude-code like thing gets input continuously from your wearable, noti…
It seems like the purpose of humans isn't to keep machines busy. When our phone or laptop is idle, it's fine if it sleeps. And when we do want something, we'd rather not wait.
(Also, this new model seems to be designed to keep latency down, which is useful for interactive tasks.)