Any MLX results?
Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
491–500 of 682 posts
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#492Earlier quoted context omitted.
Side note! Nginx was by no means the first web server to use a non-forking mechanism, nor the first open source web server to do so. Certainly Zeus (which was closed source) was earlier and very useful in this sort of application, and so was thttpd (open source, still exists as Merecat). I used thttpd quite a bit for single box applications and at one of my employers, nginx replaced a mixed strategy with Zeus, Apache…
We've barely even started on optimizations like advanced language aware grammars, and specialization routing (dynamically loading fine tunes or seperate weights for specific tasks or languages).
Right. But those still sound like modest gain territory, or qualitative gains within the same rough performance, rather than the "breakthrough" improvement notion I was responding to.
My naïve impression is that the LLM world will keep delivering these fractional improvements for some years at the cost of simplicity. And sure, ontological support seems quite promising.
But making things radically better or faster for small models in the way that is hypothesised, that feels like it can only come as a result of hardware performance improvements and likely architecture changes.
Because there's no free lunch, right? Speculative drafting for example, noticeably improves performance until acceptance rates drop for reasons that have to do with the particular application, and then it starts hurting you, especially near the limits of the memory bandwidth. Because once it is wasteful it is an extra overhead.
I gather Gemma 4 supports, in principle, dynamic speculative draft lengths, to help with this — where it will stop making bold predictions when the success rate falls. But I'm not sure if any of the inference engines I've tested with support that.
I think small models are miraculous — I still think Gemma 4 12B is astonishing — but I guess what I am saying is that I think maybe technology is moving quickly enough that the developers are done with the low-hanging fruit.
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#493I wish they would release the quantized versions in a safetensor format. Many frameworks can't load PTE and GGUF.
Try quantizing yourself, just a suggestion.
https://github.com/pytorch/executorch/tree/main/examples/mod...
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#494The post suggests that you need an rtx 5090 use it, which is currently selling for around $5,000 USD. I wouldn't exactly call that "my device", since my device costs about 25% of that for the entire computer. For the same cost, you could run on a frontier model on a pro plan for two years. The economics dont make a lot of sense for this to me, so I would love some input on why people want to do this instead (privacy,…
The post does not imply the 5090 is needed, that is just a common reference point.
A single six year old RTX 3090 works great: https://www.reddit.com/r/LocalLLaMA/comments/1vkm42m/muse_gl...
I fully expect Meta will release other, smaller Muse models in the near future too.
The 5090 is also supposed to be a $2000 GPU, not a $5000 one. The entire market is utterly distorted right now, which will impact cloud inference more and more over time too. They are not immune to the absurdly high RAM prices, so their prices will have to go up over time too until the RAM supply chain goes back to normal.
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#495[dead]
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#496The post suggests that you need an rtx 5090 use it, which is currently selling for around $5,000 USD. I wouldn't exactly call that "my device", since my device costs about 25% of that for the entire computer. For the same cost, you could run on a frontier model on a pro plan for two years. The economics dont make a lot of sense for this to me, so I would love some input on why people want to do this instead (privacy,…
I'm currently running it on an RTX 3090 (street price ~$1000 USD) with a long context and getting pretty good performance.
Prefill: ~1000 tok/s
Decode: 75-100 tok/s
It'll be far faster on a 5090, but I find the above performance to be acceptable. I've seen some claims that it even works OK on an AMD RX 7900XT (~$500USD)
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#497Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#498Earlier quoted context omitted.
brains do it with 20 watts because theyre analog. llms require massive amounts of power and this isnt changing any time soon without a breakthrough
There are arguments that the brain is quantum, as in parts of it locally using quantum effects. Which if true, might make a counter-argument, as there will be bigger data centers needed if the goal is to simulate the brain classically. On the other side, advancement in quantum computers would make current LLM inference much faster. Because of the extreme cooling needed, i dont think the energy demand would become les…
If the brain does rely on quantum effects, it's still possible the quantum effects in use are able to be simulated efficiently on a classical computer. For example if it's a matter of signal transfer rather than quantum computation, that could be simulated rather easily.
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#499Earlier quoted context omitted.
Side note! Nginx was by no means the first web server to use a non-forking mechanism, nor the first open source web server to do so. Certainly Zeus (which was closed source) was earlier and very useful in this sort of application, and so was thttpd (open source, still exists as Merecat). I used thttpd quite a bit for single box applications and at one of my employers, nginx replaced a mixed strategy with Zeus, Apache…
We've barely even started on optimizations like advanced language aware grammars, and specialization routing (dynamically loading fine tunes or seperate weights for specific tasks or languages).
[dead]
Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows
#500Will be interesting to see how Qwen3.8 27B compares against this once it releases this week. Seems like dense 30B is back in fashion? EDIT: An open weight version of Muse Spark 1.2 is going to be released as well: https://x.com/alexandr_wang/status/2086756152034066792 https://xcancel.com/alexandr_wang/status/2086756152034066792
Based on the benchmarks, it seems that Muse Glimmer barely edges out against Qwen3.6 27B, except for tool-calling skills (MCP, etc.). I wouldn't be surprised if they released it now because they are afraid they wouldn't beat Qwen3.8 27B.
I am glad they released it because I think we need a competitive culture of open weights that isn't just geopolitics.
But I have to say, I quite like the way Muse Glimmer thinks and talks. It's a cocky bastard in tone, but it's quite good, and its thinking traces are relatively terse.