Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

research.meta.ai

Post reply on HN
Print view View original

91–100 of 682 posts

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#91

Post by dannyw »

polymorph1sm wrote: post #84

Some interesting findings from the chat template designs: 1. The template name is Onyx ATEM as found in the tool call exception message 2. It appears to be following a harmony-style chat template. But the tool use seems to be a xml like : / / 3. atem: a internal joke of meta in reverse? https://huggingface.co/meta-models/Muse-Glimmer-30B/blob/mai...

The XML tags are similar to , which is obviously Anthropic ML (or ANTrophic xML).

I think it’s likely 3; meta in reverse. While tokenisers and preprocessing can catch it, you want your special tokens to be unique and not present in the original corpus. <meta: is likely too common.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#92

Post by lostmsu »

IsTom wrote: post #59
lostmsu wrote: post #32

Earlier quoted context omitted.

It seems worse than 3.6, but a bit smaller. UPD. was wrong on smaller, it's actually much larger

How is 30B smaller than 27B?

They say it is trained with quantization awareness, so it should only be 15GB or so. Qwen was only trained in FP8 with QAT.

UPD, NVM, got misled by comments here. It is actually almost 60 GB so much larger

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#93

Post by EddieRingle »

cmrdporcupine wrote: post #38
Havoc wrote: post #13

The favourable comparisons to Gemma 4 and qwen3.6 look promising!

Those two offer MoE variants, this doesn't seem to. Dense model makes it dog slow on anything without HBM. Max 15tok/sec on decode on DDR5 systems like a Spark or a Strix Halo -- and that's at 4 bit quant.

Dense models run at a very usable speed (Qwen 3.6 was running at ~50t/s last I looked) on my dual 7900 XTX desktop. (And before anyone brings it up, I did not buy them for this purpose, so the up-front cost is irrelevant in my case.)

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#94

Post by Forgeties79 »

echelon wrote: post #73
mycall wrote: post #64

Earlier quoted context omitted.

Do AI companies make release plans based on upcoming other models like this? I would think all the processes that go into the repository and weight infrastructure pre-training, checkpointing, knowledge distillation, model compression, post training pipeline, ecosystem integrations, inference API, benchmarking, human eval/safety/alignment, docs, etc... all that dictates the release schedule.

There has been a long history of AI model releases made shortly before or after a major planned release by another company. Almost always to upstage or steal thunder. Just recently, Minimax H3 released as open weights on the eve of Seedance 2.5 global availability. It's not as good, but it's good enough and it's completely open. Flux 3, which is nowhere near as good as either, suddenly announced their release once ne…

>long history

Seems a bit premature of a statement lol

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#95

Post by dist-epoch »

lisplist wrote: post #30
maxignol wrote: post #22

Optimizing speed is really the way to go. Yet 24GB is not what everyone can afford. Maybe we could take some of those 56tk/s and transfer into some free RAM space using MoE loading ? I'd be glad with a less than 10GB and more than 6tk/s model.

Unfortunately this is just the entry price for LLMs. With the exception of the Qwen 27B models, I personally haven’t found a ton of use cases for models less than 200B. With the right setup, fine tuning, etc, you can make small models do cool things, but hard to please everyone given the insane hardware costs at the moment and the comparably cheap API costs.

Gemma4-E4B (4B params) works pretty well as a local wiki, or when you don't have connectivity.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#96

Post by dannyw »

Gecko4072 wrote: post #19
127 wrote: post #12

Earlier quoted context omitted.

DSV4 Flash 0731 already runs on RTX 4090 24GB + 128GB system RAM at a usable tok/s and quantization.

You personally? Just curious. Context window is also a factor and ram isn’t really cheap. Sparks are assembled units which I like.

For the same price as a DGX Spark here (A$8499) I can buy roughly 544GB of DDR5-5200MHz from retail; which on a quad channel platform would deliver ~160gb/s real world; and ~320gb/s with octa channels (Xeon, Threadripper Pro).

If you can afford it or somehow find a used unit, you can go Epyc for 12 channels.

8/12 channel DDR5 will beat DGX Spark in inference/decode even without a GPU of any kind, as it’s memory bandwidth bound, and the Spark tops out at ~240gb/s real world.

With some optimisation and maths, it’s entirely plausible to ach

You are paying an extraordinary amount of money for the convenience of a super small unit, with still mediocre software support, but at least a community. Expect to be crawling through forum posts regularly, as SM121/Spark has many quirks and ecosystem issues still.

Please don’t pay another 70-80% gross margins on top of already inflated DRAM prices unless you need. The Spark IS really nice if you want to test out ConnectX or if you really need something small and compact and quiet.

Also consider: used Adas or even Ampere NVIDIA workstation GPUs can come with a lot of VRAM and be “reasonable”, with CUDA.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#98

Post by echelon »

Forgeties79 wrote: post #94
echelon wrote: post #73

Earlier quoted context omitted.

There has been a long history of AI model releases made shortly before or after a major planned release by another company. Almost always to upstage or steal thunder. Just recently, Minimax H3 released as open weights on the eve of Seedance 2.5 global availability. It's not as good, but it's good enough and it's completely open. Flux 3, which is nowhere near as good as either, suddenly announced their release once ne…

>long history Seems a bit premature of a statement lol

If you start counting since WaveNet or BERT, it's been ages. Especially when it feels like decades of advancements happen every single year, and rival labs are always trying to one up each other.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#99

Post by ody4242 »

zamalek wrote: post #76
formerly_proven wrote: post #60

Earlier quoted context omitted.

4K bucks buys you around 180 months of with zero upfront cost.

Problem is that might go away or get nerfed.

then you switch provider, it's not a monopoly

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#100

Post by rao-v »

cmrdporcupine wrote: post #24
Gecko4072 wrote: post #7

What I think would be perfect is a model that could run on a single DGX spark and be competitive with DSV4 Flash 731. Flash is already a game changer. Hopefully meta plans on this, like the old 70b. V4 flash is smart enough for any use but slightly too big. 27b-30b isn’t intelligent enough.

This model I think will be too slow for that on Spark, even at 4 bit quant. It's a dense model, not MoE like e.g. Qwen 35b or Gemma 4 26B A4B. On a Spark it will be memory bandwidth limited I haven't tried yet (working on it) but back of the napkin estimate puts it at around 15tok/s even after converting to NVFP4. Prefill would be much higher though. That 15tok/sec is pretty typical for dense models of this size: NVF…

Native dflash support on day 1 helps a lot! High quality speculative decoding speeds up a lot of agentic work.
Post reply on HN