Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

research.meta.ai

Post reply on HN
Print view View original

161–170 of 682 posts

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#161

Post by dannyw »

dist-epoch wrote: post #95
lisplist wrote: post #30

Earlier quoted context omitted.

Unfortunately this is just the entry price for LLMs. With the exception of the Qwen 27B models, I personally haven’t found a ton of use cases for models less than 200B. With the right setup, fine tuning, etc, you can make small models do cool things, but hard to please everyone given the insane hardware costs at the moment and the comparably cheap API costs.

Gemma4-E4B (4B params) works pretty well as a local wiki, or when you don't have connectivity.

Nitpick: Gemma4-E4B is actually a 8 billion param model, but only 4.5B params worth of memory bandwidth needed per decode.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#162

Post by xscott »

mihaelm wrote: post #26
sajithdilshan wrote: post #10

Still needs 32-64GB memory to run it locally. 64GB Macbook pro with an M5 chip costs more than 4k Euros in Germany. A more practical model would be a language specific (e.g Python or JVM language) and excellent at tool calling and reasoning. Maybe that way they can shrink it even more.

I'm sooo happy I pulled the trigger on upgrading and getting a new laptop (with 64 GB RAM) last summer. Feels like it was just in time before the exponential price jumps.

I kick myself a couple times a week for not getting the 512GB Mac Studio in February. I was holding out for an M4 or M5 chip...

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#163

Post by dannyw »

lisplist wrote: post #30
maxignol wrote: post #22

Optimizing speed is really the way to go. Yet 24GB is not what everyone can afford. Maybe we could take some of those 56tk/s and transfer into some free RAM space using MoE loading ? I'd be glad with a less than 10GB and more than 6tk/s model.

Unfortunately this is just the entry price for LLMs. With the exception of the Qwen 27B models, I personally haven’t found a ton of use cases for models less than 200B. With the right setup, fine tuning, etc, you can make small models do cool things, but hard to please everyone given the insane hardware costs at the moment and the comparably cheap API costs.

Small models are still great for lots of “simple intelligence” use cases, like annotating or summarising files and media; or even just basic chat when given web search tools.

My local NAS is private and I’m not going to send it off to APIs for captioning or metadata; but even Qwen3VL 8B does an excellent job at this, despite being quite old.

They are also really excellent for fine tuning. Unsloth and Tinker (from Mira’s TML) are great places to start.

If your use case is narrower than “coding agent for everything”, you can probably match frontier performances on that narrow domain with ~30b and exceed it with ~100b+.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#164

Post by dist-epoch »

mapontosevenths wrote: post #125
dist-epoch wrote: post #90

Earlier quoted context omitted.

There was some paper about routing at training bio-knowledge into a particular region of the model, which you then can cutoff when serving. But you probably lose some efficiency since maybe you sized that region too small/too big.

That's a very clever approach. Any idea about the papers title or authors? I'd love to look it up.

https://arxiv.org/abs/2410.04332

https://alignment.anthropic.com/2025/selective-gradient-mask...

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#165

Post by [deleted] »

jjice wrote: post #157
avaer wrote: post #101

I lament the comments saying this in any way redeems Meta (the company). The researchers releasing this stuff have almost nothing to do with Meta other than being bankrolled by the slaughterhouse. You aren't the customer, you are the pawn in big tech's game of thrones. Your good will is a commodity to be traded, almost literally. It will be used against you the moment it's convenient. This is open weights because Met…

I'd also argue this is the case for any company releasing open weights. They're not righteous, they're marketing. That's not necessarily a bad thing! They're releasing some great stuff for free and we benefit from that. Every company doing this has a motivation to not release these for free. Alibaba, Google, Moonshot, Thinking Machines, etc are not releasing their models for free because they love to. They want to gr…

[deleted]

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#166

Post by jakswa »

Another candidate for the 7900XT (20GB VRAM) I got sitting around. I pulled latest llama.cpp (targeting vulkan during build) after seeing a muse PR merged a few hours ago, and unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL runs on my 7900XT barely (and with no MTP). Sits at 19GB VRAM w/ 4 parallel 113k context slots, all layers on GPU, and at 700 tok/s prompt, and ~36 tok/s generation.

Waiting on Q3 to download to check speed + do my usual anecdotes. I generate beefy code snippets and poems, and also ingest my HOA declaration and answer nuanced questions.

edit: i should've prefaced this somewhere with: This card ballparks at 800GB/s IO, which I can't seem to find easily on the market anymore. Kinda the ideal card for this model, if I just had a _little_ more VRAM (XTX is 24GB).

edit2: not mtp, this is dflash model (param in child comment). I'm up to ~60 tok/s generation and sitting at 19GB VRAM (i added --no-mmproj (makes it text-only i believe) because I'm used to speculative decoding wanting more VRAM and I'm already close to the limit :sweat_smile:)

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#167

Post by akazantsev »

solarkraft wrote: post #11

Wow, Meta is back (at least for now)! I like this class of model. Multi-token prediction makes it viable to run dense models at not-too-far-off speeds as MoE models with much better intelligence. The submission’s title (open weights 30B local coding model) is luckily wrong: This is meant to be a general agentic model. It even comes pre-quantized and with a MTP/drafter model. Looking good! Let’s hope they aren’t disho…

> The submission’s title (open weights 30B local coding model) is luckily wrong: This is meant to be a general agentic model.

https://xcancel.com/alexandr_wang/status/2086756152034066792

It's correct. See the OpenCode demo. Generic models are good enough for coding without necessarily being designed specifically for coding.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#168

Post by dannyw »

ignoramous wrote: post #48
gunalx wrote: post #4

Meta did not abandon opensource. I would love to see a smaller distill, or a moe of this size but the benchmarks seems competetive as long as it isnt benchmaxed witch i would not be suprosed if it is.

> Meta did not abandon opensource Open weights* I don't think outside of the Big 3 (Ant, OAI, GDM), given the strong competition from China, any other Lab has a chance at capturing the coding market if they aren't open weights (save for xAI whose latest Grok looks every bit good & will probably rely on Cursor for distribution instead of going open weights). There's literally no other selling point, as the capabilitie…

Don’t sleep on NVIDIA and Nemotron.

It’s not completely open source, but they actually release their pretraining and post-training datasets with some redactions for (cough) pirated content.

They also have very good code and playbooks for actually doing a fine-tune, CPT, etc.

Even if you’re not tuning a Nemotron model, its mixes are very excellent for your replay data slice; or general experiments. Way better curation and quality than Dolma, etc; or other large huggingface data mixes I tested.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#169

Post by zImPatrick »

LorenDB wrote: post #149
monster_truck wrote: post #119

Earlier quoted context omitted.

Meta can never be redeemed, but it's still valid to admit that FB at one point had a very badass engineering culture. They're one of 2 companies I would absolutely never work for (weapons etc aside). FB's recruiters hounded me so often I requested that they blackball me. The day they became Meta, I learned this by checking my email to see that they started trying to reach out again. I once again requested that they b…

What is the other company that you would never work for?

He said OpenAI in the comment (if I read it correctly)

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#170

Post by Der_Einzige »

skinfaxi wrote: post #120
avaer wrote: post #101

I lament the comments saying this in any way redeems Meta (the company). The researchers releasing this stuff have almost nothing to do with Meta other than being bankrolled by the slaughterhouse. You aren't the customer, you are the pawn in big tech's game of thrones. Your good will is a commodity to be traded, almost literally. It will be used against you the moment it's convenient. This is open weights because Met…

How is this non-sequitor the top comment?

This is par for the course, HN is far worse than reddit on balance, especially involving upvoting/downvoting decorum.

Go vibecode something to auto upvote all downvoted posts, call it "Antiechochamber.HN" or something, and if enough people used it this website might improve a bit.

Post reply on HN