Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

research.meta.ai

Post reply on HN
Print view View original

571–580 of 682 posts

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#571

Post by arkmm »

Computer0 wrote: post #546
lukaslalinsky wrote: post #543

Earlier quoted context omitted.

This is is already possible with Claude Code. I use a setup where I have one instance monitoring a local queue, I have a web app for receiving webhooks from various sources and pushing them to the queue. Plus email for things that don't have webhooks. That instance then decides what to do with each input, sometimes it can spawn additional agent to investigate/prepare, sometimes it creates a ticket assigned to me and…

If you are willing and not too busy, What model do you use and what is your cost? (If using subscription would you be able to check with 'npx ccusage').

I'd also be really curious about the cost to run something like this, and what things you think it's particularly helpful for?

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#572

Post by andy99 »

dofm wrote: post #565
andy99 wrote: post #562

Earlier quoted context omitted.

Having spent a good part of the day with it, glimmer reminds me of Rorschach from The Watchmen. No unessential parts of speech, action oriented, brief and to the point. From a token perspective anyway it’s great, and it seems to hold its own well against more verbose models. I really do feel like it’s effective tok / s is way higher because it doesn’t waste them.

I am very struck by the way open weights LLMs seem to reflect a culture. I don't really enjoy the way Qwen writes prose, and I find its thinking a bit exhausting, though it clearly writes very good code. I like the neutral, clear way the Gemma models write, which I sometimes use to get myself a "getting started" document on something I want to understand; it also summarises well. It is neutral, sensible, un-showy. It…

I think we’re going to see a lot more “product“ focus in the future with deliberate attention paid to these kind of properties. Historically though there are some obvious differences, the focus has been on benchmark maximizing. As that saturates, I expect more interesting choices about writing and thinking style designed to be differentiators instead of a side effect. Kudos to the PM here for taking it in a different directions, there’s obviously been thought put into it.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#573

Post by hadlock »

dofm wrote: post #548
0xc133 wrote: post #544

Earlier quoted context omitted.

I was able to use rope/yarn scaling with llama.cpp to extend the context window to 256K and it seems to be pretty usable on the debugging-and-bugfixing session I have that’s 216K tokens deep. No infinite output loops, reasoning is still coherent, tool calls appear to be passing and failing at roughly the same rate as a fresh context window. Haven’t tried going to 512K or higher yet but 256K definitely seems usable to…

Useful stuff, thank you. And more reading for me to do! (DFlash seems rather weird on my M1 Max; model performance suddenly drops off a cliff like maybe there is some memory management issue).

In my experience I am getting 23-24 t/s output with dflash off, and it craters to ~9 t/s with it on, miss rate exceeding 50%. And I'm using the same device as stated on their model page/card. We might need to wait for the software to catch up

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#574

Post by bitexploder »

skohan wrote: post #388
exe34 wrote: post #376

Earlier quoted context omitted.

Thanks for that, it's the level I like to work too - what model/quant are you using? How much vram/context and which coder?

I'm using Qwen3.6 27B Q4, max context with pi on 32GB VRAM (although I'm testing out Glimmer on a feature implementation literally right now). Pi is great because it has minimal context added by the agent. Looking forward to the 3.8 27B release to compare.

Pi and a similar set of tools is also likely similar to the harness these models are trained on. More complex harnesses burn reasoning tokens on these small models and in my benchmarking don't seem to be able to beat Pi ever. Usually it isn't close on some tests.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#575

Post by opensandwich »

dofm wrote: post #548
0xc133 wrote: post #544

Earlier quoted context omitted.

I was able to use rope/yarn scaling with llama.cpp to extend the context window to 256K and it seems to be pretty usable on the debugging-and-bugfixing session I have that’s 216K tokens deep. No infinite output loops, reasoning is still coherent, tool calls appear to be passing and failing at roughly the same rate as a fresh context window. Haven’t tried going to 512K or higher yet but 256K definitely seems usable to…

Useful stuff, thank you. And more reading for me to do! (DFlash seems rather weird on my M1 Max; model performance suddenly drops off a cliff like maybe there is some memory management issue).

As an extra data point, I have exactly the same issue on a M1 Max as well.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#576

Post by colingauvin »

jawiggins wrote: post #538

> Muse Glimmer is a 30-billion-parameter model optimized for always-on local agent workflows. It’s small enough to run on a Mac or PC with a single consumer GPU, enabling use cases that range from local agents and function calling, to local coding, and LLM-as-a-judge evaluation. The next iteration in LLM products is a 24/7 thinking loop where the claude-code like thing gets input continuously from your wearable, noti…

I've been building this for the last 6 months or so. I've basically got it working. The model is not the issue, the infra is. Keeping everything in context just isn't possible and LLMs, even Fable, don't mode switch well. To get around this I've built a database software that ingests as much digital information as possible, and annotates it, then creates timelines with resolution gradients (longer ago = less resolution) that it feeds to the LLM on every request.

Then you have your cheap little MoE or ternary model just running in a loop, with an escalation pathway before it reaches the big expensive models.

Currently it's doing things like reminding me to take allergy medication when I wake up because it's checked AQI or whatever, reminding me to stop at the market when I'm on my way to pick up the kids to get the cherry tomatoes I forgot, giving me heads up of what folks are expecting from me in certain meetings based on cross correlating email and calendar, etc.

It's honestly the single most productive tool I've found for my ADHD.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#577

Post by nateb2022 »

colingauvin wrote: post #576
jawiggins wrote: post #538

> Muse Glimmer is a 30-billion-parameter model optimized for always-on local agent workflows. It’s small enough to run on a Mac or PC with a single consumer GPU, enabling use cases that range from local agents and function calling, to local coding, and LLM-as-a-judge evaluation. The next iteration in LLM products is a 24/7 thinking loop where the claude-code like thing gets input continuously from your wearable, noti…

I've been building this for the last 6 months or so. I've basically got it working. The model is not the issue, the infra is. Keeping everything in context just isn't possible and LLMs, even Fable, don't mode switch well. To get around this I've built a database software that ingests as much digital information as possible, and annotates it, then creates timelines with resolution gradients (longer ago = less resoluti…

Think of an LLM as a thesaurus, but for entire trains of thought rather than words. Your initial query yields something pertinent to the task at hand. But let it endlessly recurse and... you end up with something completely useless.

People would do well to acquire at least a modest familiarity with what an LLM actually is. NLP is fascinating. So is entropy.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#578

Post by Grombobulous »

dannyw wrote: post #141
Grombobulous wrote: post #122

Earlier quoted context omitted.

I think it’s also worth pointing out that that there are numerous less evil options to choose from. Perhaps none of the AI companies are shining examples of high ethics, but basically all of them have ethical high ground over Meta. At least Anthropic isn’t sending private videos from pervert glasses to contract workers in Africa. It’s a low bar but it’s a bar nonetheless.

I personally don’t like, and wouldn’t work for Meta; but it’s an Apache 2.0 model. I’m liking it, and I don’t see a personal moral contradiction here. Do you use React for frontend for example? I also wish this HN post is a bit more focused on the release, and less noise around Meta.

There’s no moral contradiction, but there is a moral decision.

I don’t use React for front end nor do I willingly use any Meta product.

I would like to focus on the release of the Volkswagen Beetle and hear less noise about Adolf Hitler.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#579

Post by usef- »

dofm wrote: post #351
nbardy wrote: post #343

Earlier quoted context omitted.

Everyone keeps repeating this who doesn’t understand the underlying technology. Small llms are still way more efficiently server on big GPUs. Sharing server capacity takes advantage of the massive parallel throughput and sharing of memory bandwidth. You are sharing the GPUs with thousands of concurrent users.

FWIW it is entirely possible to square the notion that small models will still be hosted on cloud hardware with the idea that the data centre buildout will end in tears. Many analysts (and Microsoft) think even now that if everything committed gets built there will be considerable oversupply and there is not the revenue to pay for it. If small models do continue to improve in unusual ways (I think there are limits) t…

So far everyone seems to be consistently GPU-poor, despite the huge buildout, and usage keeps going up drastically. I don't know what would make usage drop.

Every time they've made smarter models we've wanted the smarter ones, and local models runnable on typical hardware are still very far behind in speed and intelligence (as neat as they are)

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#580

Post by HDBaseT »

notnullorvoid wrote: post #498
Arwill wrote: post #459

Earlier quoted context omitted.

There are arguments that the brain is quantum, as in parts of it locally using quantum effects. Which if true, might make a counter-argument, as there will be bigger data centers needed if the goal is to simulate the brain classically. On the other side, advancement in quantum computers would make current LLM inference much faster. Because of the extreme cooling needed, i dont think the energy demand would become les…

If the brain does rely on quantum effects, it's still possible the quantum effects in use are able to be simulated efficiently on a classical computer. For example if it's a matter of signal transfer rather than quantum computation, that could be simulated rather easily.

I wonder what company will release the first "quantum LLM"
Post reply on HN