Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

research.meta.ai

Post reply on HN
Print view View original

661–670 of 675 posts

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#661

Post by colingauvin »

docjay wrote: post #659
colingauvin wrote: post #655

Earlier quoted context omitted.

I sort of do. For scheduled things I am using Goose and subagents per task. So I wake up at 6 am to a briefing that was composed by a ton of different agents. That works well in a narrow programmatic setting, but it breaks down in certain natural conversation context. For instance voice control of something like "text my sister that I'm running late" - self hosted models are terrible at figuring out who my sister is.…

I was afraid my description sounded like agents, and it kind of is, but not like most implementations. Most use a “boss bot” to craft a prompt/system message and launch the model, and sometimes they redo it every time it launches the agent. It’s a low effort attempt that’s immediately flawed because it uses LLM output for LLM input. It can look like it’s working for some time, but the perplexity guarantees it’s a rol…

>Run the same exact prompt 100x in a single step test (one prompt, one response), hash the full responses, and you’ll see 10-50+ unique responses. The higher the unique the worse your prompt; focus on the system. A highly tuned system prompt will result in one response, even at a temperature of 1.0. Really. Once that’s done that’s the only thing it does and it’s the only one that does it and it never changes. Other LLMs that call ‘check_email()’ are unknowingly just passing a prompt to the specialized one.

This is a brilliant idea, thank you. Convergence as a metric for prompt robustness.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#662

Post by christkv »

scrlk wrote: post #5

Will be interesting to see how Qwen3.8 27B compares against this once it releases this week. Seems like dense 30B is back in fashion? EDIT: An open weight version of Muse Spark 1.2 is going to be released as well: https://x.com/alexandr_wang/status/2086756152034066792 https://xcancel.com/alexandr_wang/status/2086756152034066792

Is qwen confirmed to release?

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#663

Post by kevincox »

alfiedotwtf wrote: post #657
kevincox wrote: post #588

Earlier quoted context omitted.

For me Laguna frequently slightly corrupted text then it would be unable to notice the difference and get stuck making the dumbest conclusions. Thinks like typoed directory or function names. It was a great model other than that, but I ended up just going back to Qwen3.6

Make sure you've got the latest updates - both Poolside and Unsloth updated about a week after release, and that you're running via llama.cpp (I've noticed on the same model and same command line arguments, ik_llama.cpp loses quality for me). Also just in case you didn't see it, Laguna S 2.1 was having issues on MXPF4 but people on Q4 and Q6 couldn't repeat the problem.

I just downloaded the latest llama.cpp and poolside Q4 and within 10k tokens it corrupted "Use ~/p/kevincox.ca as a reference" into "kevinc.ai" and then starts doing stupid things because the directory "doesn't exist".

In previous testing the Q8 was better but still had the same problem frequently enough to be very difficult to use. I don't know if it is a runner bug or model bug but at least with llama.cpp this seems to always happen and I haven't seen an issue like this with any other model.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#664

Post by fwipsy »

dofm wrote: post #515
notnullorvoid wrote: post #513

Earlier quoted context omitted.

The gains wouldn't be "free lunch", it's the result of time and effort researching optimal design and architecture. Even if the idea of "no free lunch" was taken liberally discounting the cost of research, it would only be limiting to systems built from a foundation of optimization, but that's not the case. The foundation so far has been one of brute force scaling. Usually meaning there is lots of room for optimizati…

> The foundation so far has been one of brute force scaling. In cloud AI, sure. But in the smaller open weights model territory it feels like we're well into optimisation?

Why do you think cloud AI doesn't also use these optimizations? They are also compute-constrained. I expect optimization and scale have a multiplicative effect. Surely all the serious players care about both.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#665

Post by drob518 »

jfrbfbreudh wrote: post #460
drob518 wrote: post #203

Earlier quoted context omitted.

I was about a week away from buying a very tricked out MacBook Pro with 128 GB RAM, but was on vacation and worried about it arriving while I was away, and then the price hikes went into effect. Grumble. Oh, well. Serves me right.

https://www.ign.com/articles/ramageddon-continues-another-ye... The best time to plant a tree…

Indeed. Sigh.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#666

Post by alfiedotwtf »

kevincox wrote: post #663
alfiedotwtf wrote: post #657

Earlier quoted context omitted.

Make sure you've got the latest updates - both Poolside and Unsloth updated about a week after release, and that you're running via llama.cpp (I've noticed on the same model and same command line arguments, ik_llama.cpp loses quality for me). Also just in case you didn't see it, Laguna S 2.1 was having issues on MXPF4 but people on Q4 and Q6 couldn't repeat the problem.

I just downloaded the latest llama.cpp and poolside Q4 and within 10k tokens it corrupted "Use ~/p/kevincox.ca as a reference" into "kevinc.ai" and then starts doing stupid things because the directory "doesn't exist". In previous testing the Q8 was better but still had the same problem frequently enough to be very difficult to use. I don't know if it is a runner bug or model bug but at least with llama.cpp this seem…

Wow that's weird. I have it running sometimes at least 8 hours straight on a single prompt and no weirdness.

Could it be something weird like you're hitting your compaction limit?

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#667

Post by kevincox »

alfiedotwtf wrote: post #666
kevincox wrote: post #663

Earlier quoted context omitted.

I just downloaded the latest llama.cpp and poolside Q4 and within 10k tokens it corrupted "Use ~/p/kevincox.ca as a reference" into "kevinc.ai" and then starts doing stupid things because the directory "doesn't exist". In previous testing the Q8 was better but still had the same problem frequently enough to be very difficult to use. I don't know if it is a runner bug or model bug but at least with llama.cpp this seem…

Wow that's weird. I have it running sometimes at least 8 hours straight on a single prompt and no weirdness. Could it be something weird like you're hitting your compaction limit?

No compaction happened. (That is set near 256k.) It seems to really not like my domain but I've seen it for various things as well (especially high-entropy tokens). Dropping random characters or various things. I've tried various flash attention and context quantization settings but nothing seemed to resolve it completely.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#668

Post by nicman23 »

mmaunder wrote: post #249

Remember when we needed 200 servers for an enterprise website because Apache used one process or thread per connection - and Nginx collapsed that into a single box overnight? That moment for LLMs is near. It’s going to move us from the big iron era of AI to small portable brains. Nature has already proved it’s possible with 20 watts and very little heat generation. And I think the data center buildout will end in car…

qwen 3.6 27b is already there for me to be honest

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#669

Post by mmaunder »

semiquaver wrote: post #542
mmaunder wrote: post #249

Remember when we needed 200 servers for an enterprise website because Apache used one process or thread per connection - and Nginx collapsed that into a single box overnight? That moment for LLMs is near. It’s going to move us from the big iron era of AI to small portable brains. Nature has already proved it’s possible with 20 watts and very little heat generation. And I think the data center buildout will end in car…

I don’t remember that and I was there! The idea that the performance delta between Apache and nginx for any normal workload is anything like 20,000% is absurd.

Oh my. Funny story. Kerry and I (the Wordfence founders) met at eToys dot com back in the day and she’s now my wife of 20 years. Anyway at eToys she ran the web QA team and I ran the warehouse management system in Europe. At eToys we had over 200 front end servers (for the memory needed for the number of concurrent processes needed to support the number of connections) and later that increased IIRC. We got together and started a string of startups. Feedjit was one of them and it would get over 1 billion modperl hits in a month on over 1M websites. We scaled it up with just rhe two of us with 6 servers using modperl with Apache on the back end and Igor released nginx just as we hit exponential growth removing our need to scale horizontally. Suddenly we could have a single front end box with nginx, reenable keepalive, and I had 150,000 concurrent tcp connections hanging off a single Linux box thanks to nginx and epoll in the Linux kernel. Massive game changer. The power of event based design vs threaded or multi process design.

Not absurd. Sometimes stair steps in tech do happen. And they will in AI.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#670

Post by skybrian »

ajam1507 wrote: post #650
skybrian wrote: post #550

Earlier quoted context omitted.

Maybe it's my lack of imagination, but what do you imagine you'd be doing where you'd want to keep a computer busy overnight? It seems like the purpose of humans isn't to keep machines busy. When our phone or laptop is idle, it's fine if it sleeps. And when we do want something, we'd rather not wait. (Also, this new model seems to be designed to keep latency down, which is useful for interactive tasks.)

>It seems like the purpose of humans isn't to keep machines busy. When our phone or laptop is idle, it's fine if it sleeps. And when we do want something, we'd rather not wait. I'm not sure what the original commenter had in mind, but just because our machines are idle when we aren't using them doesn't mean that, that's how we will use computers in the future. I think notifications are an example even now of the comp…

Notifications are designed to be low-overhead, though. I’d still like to know of specific examples for why it’s worth making your computer go burr all night, rather than asserting it will be that way just because you could.
Post reply on HN