Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

research.meta.ai

Post reply on HN
Print view View original

601–610 of 682 posts

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#601

Post by ElectricalUnion »

cyanydeez wrote: post #570
wronglebowski wrote: post #8

Earlier quoted context omitted.

It’s really interesting timing, Qwen over thinking is what kills it for me. I’m just glad we have more options in this size class now.

Llamscpp provides reasoning budget and message. You can use the message to redirect it. Once you get the agent and message consistent,itll keep moving.

You can use any message you want, but the model was tested to react reasonably well to the specific token sequence of "\nConsidering the limited time by the user, I have to give the solution based on the thinking directly now.\n.\n\n" (from a Alibaba paper, struggling to find it now)

Edit: arXiv:2505.09388 Qwen3 Technical Report

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#602

Post by computershit »

rancor wrote: post #533
computershit wrote: post #475

Earlier quoted context omitted.

I use llama.cpp w/ llama-swap https://github.com/ggml-org/llama.cpp https://github.com/mostlygeek/llama-swap

FYI, llama-server can now be run in router mode so llama-swap is probably only needed for more exotic scenarios.

I’m using llama-swap because it can manage arbitrary backends, not just llama-server instances. I have llama.cpp chat and embedding models running alongside whisper-server all behind a single endpoint with per-model TTLs so they don't fight over the limited vram I have available on this box. Native routing could replace the llama.cpp part but not whisper so I guess I'm exotic ;)

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#603

Post by rco8786 »

mmaunder wrote: post #249

Remember when we needed 200 servers for an enterprise website because Apache used one process or thread per connection - and Nginx collapsed that into a single box overnight? That moment for LLMs is near. It’s going to move us from the big iron era of AI to small portable brains. Nature has already proved it’s possible with 20 watts and very little heat generation. And I think the data center buildout will end in car…

> Nature has already proved it’s possible with 20 watts and very little heat generation

Never thought of it in those terms before.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#604

Post by nojs »

pu_pe wrote: post #46
scrlk wrote: post #5

Will be interesting to see how Qwen3.8 27B compares against this once it releases this week. Seems like dense 30B is back in fashion? EDIT: An open weight version of Muse Spark 1.2 is going to be released as well: https://x.com/alexandr_wang/status/2086756152034066792 https://xcancel.com/alexandr_wang/status/2086756152034066792

Based on the benchmarks, it seems that Muse Glimmer barely edges out against Qwen3.6 27B, except for tool-calling skills (MCP, etc.). I wouldn't be surprised if they released it now because they are afraid they wouldn't beat Qwen3.8 27B.

Qwen3.6 27B has really punched above its weight for a long time. It’s shockingly good for its size. Very excited to see what 3.8 can do.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#605

Post by segmondy »

mmaunder wrote: post #249

Remember when we needed 200 servers for an enterprise website because Apache used one process or thread per connection - and Nginx collapsed that into a single box overnight? That moment for LLMs is near. It’s going to move us from the big iron era of AI to small portable brains. Nature has already proved it’s possible with 20 watts and very little heat generation. And I think the data center buildout will end in car…

20 watts huh? How much energy has nature sure is required for lift? Does the same amount of energy scale by size for airplanes?

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#606

Post by rhplus »

jermaustin1 wrote: post #449
toyg wrote: post #369

Earlier quoted context omitted.

Is there some sort of dedicated tool for this type of setup, or did you hand-craft it ?

Somewhat hand rolled, somewhat claude coded. Back in 2023 I started my own C# LLM library for doing tool calls and structured output, and over the years it has morphed bigger and bigger, and that is the backbone of almost all of my LLM-based projects. I've never released it, but its easy to understand, and simple to add your own tools: [AIDescription("Get current weather for a location")] static string GetWeather( [A…

It looks like you’re describing something like an MCP server and a client model.

If you’re in the C# ecosystem you could consider converting your APIs to MCP format tools using the MCP SDK.

https://devblogs.microsoft.com/dotnet/build-a-model-context-...

And then leveraging Microsoft Agent Framework for the client and orchestration side of things:

https://learn.microsoft.com/en-us/agent-framework/

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#607

Post by segmondy »

GodelNumbering wrote: post #209

https://xcancel.com/finkd/status/2086755195535413696 "... Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model..." This is bigger news - good for self hosting enthusiasts and a strategically sound move for Meta. Any push towards 'anti Chinese' models will directly benefit Meta as the competition on the frontier open-weights American models is almost non-existent. Meta will have no probl…

Yeah, but if it's huge, how many can run it? Many folks struggle to run 200B+ models

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#608

Post by segmondy »

andy99 wrote: post #327

The gguf is up and works, I don’t know if it’s them or unsloth that’s facilitated this but it’s nice because e.g. Inkling still doesn’t appear to have support in llama.cpp which makes it irrelevant to a class of user. Unfortunately I don’t have enough experience with Qwen 27B to immediately compare, but I do it’s Qwen 3.6 35B A3. It’s much slower obviously but it seems to be way more efficient with its thinking to th…

There's an inkling branch, go to unsloth, read - https://unsloth.ai/docs/models/inkling

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#609

Post by dannyw »

segmondy wrote: post #607
GodelNumbering wrote: post #209

https://xcancel.com/finkd/status/2086755195535413696 "... Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model..." This is bigger news - good for self hosting enthusiasts and a strategically sound move for Meta. Any push towards 'anti Chinese' models will directly benefit Meta as the competition on the frontier open-weights American models is almost non-existent. Meta will have no probl…

Yeah, but if it's huge, how many can run it? Many folks struggle to run 200B+ models

Two DGX sparks will run the full Deepseek V4 Flash full. (It is definitely expensive, but relatively easy and compact; and extremely power efficient)

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#610

Post by _kb »

dofm wrote: post #351
nbardy wrote: post #343

Earlier quoted context omitted.

Everyone keeps repeating this who doesn’t understand the underlying technology. Small llms are still way more efficiently server on big GPUs. Sharing server capacity takes advantage of the massive parallel throughput and sharing of memory bandwidth. You are sharing the GPUs with thousands of concurrent users.

FWIW it is entirely possible to square the notion that small models will still be hosted on cloud hardware with the idea that the data centre buildout will end in tears. Many analysts (and Microsoft) think even now that if everything committed gets built there will be considerable oversupply and there is not the revenue to pay for it. If small models do continue to improve in unusual ways (I think there are limits) t…

When the glut of GPU arrives I'm sure humanity will find a good use for all that excess compute, like finally getting back to signing monkey pictures and excreting endless hash based pyramid schemes.
Post reply on HN