Live data from Hacker News

Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

research.meta.ai

441–450 of 682 posts

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#441

Earlier quoted context omitted.

I've been using Qwen3.6 35B A3B, and with reasoning turned on, I'd say 2/3 (give or take) of the tokens for a response are thinking tokens. Which at 70+ tps locally, that isn't that awful. I run an 80k context across 4-10 "agents" for my solo TTRPG, where Qwen is the GM, each NPC at a location, the director, and the narrator. Each turn is about 45-60 seconds to generate all of the various responses. The GM and direct…

Are you running inference in parallel? 70 tps seems low for parallel execution.

It is on a single 3090, and that seems to be where it averages out. I'll get 85tps on turn 0, but then it settles down to low 70s within a few turns, but holds steady at that.

My issue currently is KV Cache, because I can't keep enough parallel caches running (4 is where I'm at), so TTFT (is that the initialism?) can be long when I have a particularly large scene (basically more than 2 NPCs).

But my harness does let me offload to any OpenAI compatible endpoint, I just prefer local cuz free.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#442

Earlier quoted context omitted.

When you say you are anti-American, what do you mean by that? Are you, for example, wishing for the demise of the United States? Do you want to tear down the 1st Amendment and the Statue of Liberty? Are you against democracy? Do you want our businesses and factories to shut down and go out of business? Are you willing or would you support foreign countries attacking our military at home and abroad? Are you cheering a…

> Are you, for example, wishing for the demise of the United States? Do you want to tear down the 1st Amendment and the Statue of Liberty? Are you against democracy? Do you want our businesses and factories to shut down and go out of business? Are you willing or would you support foreign countries attacking our military at home and abroad? Are you cheering against our athletes? I am very pro the "dream" of America, i…

> I am very pro the "dream" of America, in terms of liberty, democracy, etc. According to every "democracy index" I'm aware of we're not doing so hot in that regard, generally rating as a flawed/deficient democracy and the trends are going in the wrong direction, fast.

Well, to be fair people do have different dreams. I'm not sure those indices count for a whole lot. As an example, folks who argue in favor of returning more power to the individual states are, certainly, acting in accordance with stronger democratic principles. I'd argue the EU is actually a bit anti-democratic as it removes more power from local populations and individual states/countries/entities. Yet how would individual democracy indices rate these two?

And if you disagree with my perspective on what is more democratic, well, who is right and who is wrong?

> Generally, no, but this is way too open-ended of a question. I want good economic opportunity for everyone, including every US citizen. But relevant to the OP if Meta got snapped out of existence I think it would be a net positive for the world.

I'm no fan of Meta. But they aren't the only entity where if they snapped out of existence it would be a net positive for the world. I can think of a few non-corporate entities at least.

But fair enough it's a bit open-ended. I guess if push comes to shove when you say you are anti-American do you want to see, for example, economic opportunity increase in other countries at the expense of Americans? Not all relations have to be such give-and-take, but certainly many economic opportunities are. I'm just trying to learn more about what being an anti-american means to you.

> Nope. But across our entire history ask yourself how many foreign countries have attacked the US? Now ask yourself how many the US has attacked. With those numbers in mind, does the US seem like "the good guys"? really? ...really?

I'm fine doing things like removing all of our overseas bases, leaving alliances including NATO, never again being involved in the affairs of other countries, whether that's Korea (Korean War), Kuwait, Bosnia, or Iran, or Taiwan and China, and cutting our military spending quite a bit to reflect your desire to not be the bad guys anymore. I say let's just sit back and let the rest of the world figure out their problems - why do we need to be involved?

Is that something you agree with as well? Would that be anti-American in alignment with your views or is there other nuance here I'm not properly capturing?

> Nope, but I'm not cheering for them either just because they are American, I'm not a tribalist.

And to be clear even if America was even, say, your ideal dream since you're not a tribalist you wouldn't cheer on American athletes and would also label (not as a matter of insult but as fact) other people who cheer on their compatriots as tribalist?

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#443

Earlier quoted context omitted.

You personally? Just curious. Context window is also a factor and ram isn’t really cheap. Sparks are assembled units which I like.

I've run it with a large context window on 256GB ram + 4090. It wasn't super fast, but it was manageable and it completed the tasks I gave it well.

How many tps and size of ctx?

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#444

Earlier quoted context omitted.

I've been using Qwen3.6 35B A3B, and with reasoning turned on, I'd say 2/3 (give or take) of the tokens for a response are thinking tokens. Which at 70+ tps locally, that isn't that awful. I run an 80k context across 4-10 "agents" for my solo TTRPG, where Qwen is the GM, each NPC at a location, the director, and the narrator. Each turn is about 45-60 seconds to generate all of the various responses. The GM and direct…

This sound very interesting, do you have any resource I could look at? Me and my son did a very rudimentary (compared to yours) setup to play Paranoia, but this is at another level.

I'll go over my repo, and see if it is hiding any API keys and maybe make it public. The issue I have is it relies on a nuget package that also isn't live (its in my local nuget feed).

I'm not sure what all is needed to make that work for people.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#445
post #152

Earlier quoted context omitted.

It’s really interesting timing, Qwen over thinking is what kills it for me. I’m just glad we have more options in this size class now.

Qwen thinking is really good in Mandarin; and probably natively trained the most there. Try a system prompt requiring it to think in Mandarin, while still delivering the response in the user’s language.

Is the quality of the thinking better or it's just shorter since Mandarin is more compact?

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#446

Earlier quoted context omitted.

It’s really interesting timing, Qwen over thinking is what kills it for me. I’m just glad we have more options in this size class now.

Just to play devil’s advocate: you can’t compare Qwen to a (proprietary/closed source) hosted model and deduce that Qwen is overthinking, as Qwen gives you the full reasoning/thinking trace while all the proprietary models now give you only a summary “to prevent distillation”, making it hard to properly compare apples to apples here.

You can tell how long the cloud models spend thinking based on the delay.

The Qwen models have a habit of going into thought loops where they go in circles for a while.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#447

Earlier quoted context omitted.

When you say you are anti-American, what do you mean by that? Are you, for example, wishing for the demise of the United States? Do you want to tear down the 1st Amendment and the Statue of Liberty? Are you against democracy? Do you want our businesses and factories to shut down and go out of business? Are you willing or would you support foreign countries attacking our military at home and abroad? Are you cheering a…

I can't speak to the parent commenter's precise meaning, but I interpreted them to mean they oppose the current domestic and foreign policy of US political and business leadership (a distinction that blurs to the point of indistinguishability of late, courtesy of said domestic policy). I believe that many Americans that were previously dismissive or ambivalent regarding critiques of US activity at home and abroad (ei…

Yea but if they just oppose the current administration isn't that just being anti-Trump instead of anti-American? His administration is just some other administration that'll come and go like foul sewage gas. We've had 47 of them. Does the OP become pro-America when their party is in charge or when certain conditions are met?

I don't know, just asking.

But I know that in my own upbringing I've always viewed America as a place where, because it's a democracy, we are never going to all be aligned or on the same page about policy direction. Even when I disagree with the Biden Administration or the Trump Administration on some number of issues, and believe my I've got a lot of those issues with the current one, I wouldn't think of myself as anti-American or wishing harm to the country. But that's just my own experience.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#448

Earlier quoted context omitted.

> Only China can release good, open weight models and American companies can’t compete. Oh by the way all the spend is for nothing because China alone can release open-weight models thus destroying American AI. Let me be blunt and let me say: you don't understand why we people support Chinese models. 1. Chinese labs started with open weight models, US labs started with dooms day narrative 2. US VC based companies mus…

US companies published their research on transformers. The only reason Chinese companies have any models at all is open US research... Add distillation to that, and the entire Chinese AI sphere is really just an offshoot of public US AI information...

All LLM’s are distillation of stolen copyrighted materials, some of which are the outputs of other LLM’s.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#449
post #369

Earlier quoted context omitted.

I've been using Qwen3.6 35B A3B, and with reasoning turned on, I'd say 2/3 (give or take) of the tokens for a response are thinking tokens. Which at 70+ tps locally, that isn't that awful. I run an 80k context across 4-10 "agents" for my solo TTRPG, where Qwen is the GM, each NPC at a location, the director, and the narrator. Each turn is about 45-60 seconds to generate all of the various responses. The GM and direct…

Is there some sort of dedicated tool for this type of setup, or did you hand-craft it ?

Somewhat hand rolled, somewhat claude coded.

Back in 2023 I started my own C# LLM library for doing tool calls and structured output, and over the years it has morphed bigger and bigger, and that is the backbone of almost all of my LLM-based projects.

I've never released it, but its easy to understand, and simple to add your own tools:

  [AIDescription("Get current weather for a location")]
  static string GetWeather(
    [AIDescription("The city name")] string city,
    [AIDescription("The country name")] string country,
    [AIDescription("Temperature unit", ["C", "F"])] string unit = "C")
  {
    // make some API call to a weather API and return a string to the LLM
    return $"The weather in {city}, {country} is 22°{unit} and sunny";
  }
  
  var chat = client.StartConversation("You are a helpful assistant with access to weather data.");
  var response = await chat.SendAsync("What's the weather in London?", GetWeather);
I'm sure plenty of better libraries exist for this now, but in 2023, I don't think any existed in the dotnet ecosystem. I've never released it though, because I've never "finished" it.

Re: Muse Glimmer: 30B-parameter model optimized for always-on local agent workflows

#450

Earlier quoted context omitted.

I'm a 52 year old natural born US citizen whose ancestors have been here for generations and I'm currently anti-American. Why wouldn't I be? We've never been the shining beacon of light we would claim to be, but we're so fucking awful now.

When you say you are anti-American, what do you mean by that? Are you, for example, wishing for the demise of the United States? Do you want to tear down the 1st Amendment and the Statue of Liberty? Are you against democracy? Do you want our businesses and factories to shut down and go out of business? Are you willing or would you support foreign countries attacking our military at home and abroad? Are you cheering a…

Why would you tear down a gift from France?
Post reply on HN