Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Inference Optimization for MiMo v2.5: Pushing Hybrid SWA Efficiency to the Limit

mimo.xiaomi.com

Post reply on HN
Print view View original

31–40 of 53 posts

Re: Inference Optimization for MiMo v2.5: Pushing Hybrid SWA Efficiency to the Limit

#31

Post by throwa356262 »

polski-g wrote: post #7
limbicsystem wrote: post #6

Earlier quoted context omitted.

Their pricing is incredible on the token plan - something like 50b tokens for $60!

No that's not right. It's 50b credits, not tokens. What is a credit? Nobody knows. Someone did the math a few months ago and paying API prices was the same as the monthly subscription.

> Nobody knows

Now you know:

2.5 pro: 300/600 credits per input/output token

2.5: 100/200

Cached tokens are 2-3 credits.

If you only use pro, with a 7/1 ratio and no discounts or penalties the $6 plan gets you a total of 12M tokens. This assumes zero cached tokens thought.

Re: Inference Optimization for MiMo v2.5: Pushing Hybrid SWA Efficiency to the Limit

#32

Post by embedding-shape »

nmfisher wrote: post #28

Related, I was given access to mimo-v2.5-ultraspeed, which is amazing. This is now my expectation for speed, it’s fast enough for me to stay mentally engaged rather than getting distracted waiting for the agent to churn.

The -spark variant of GPT was a ton of fun indeed, such a shame it's so dumb though so really hard to rely on. If you were to compare the quality of mimo-v2.5-ultraspeed with anything from OpenAI/Anthropic, where would it be placed ~more or less in your view?

Re: Inference Optimization for MiMo v2.5: Pushing Hybrid SWA Efficiency to the Limit

#33

Post by AgentMasterRace »

trollbridge wrote: post #25
TurdF3rguson wrote: post #19

Earlier quoted context omitted.

That's not really true though, providers are deprecating models and I have at least 10 emails to prove it.

A provider deprecating a model doesn't mean the .gguf file disappears from my computer.

hell yeha bro, I still rock qwen 0.1

Re: Inference Optimization for MiMo v2.5: Pushing Hybrid SWA Efficiency to the Limit

#34

Post by chronogram »

TurdF3rguson wrote: post #27
trollbridge wrote: post #24

Earlier quoted context omitted.

Because I can run Qwen 3.6 or DeepSeek V4 until the end of time if I want to? The model is on HuggingFace; anyone can download it. I have Qwen and Gemma on my laptop right now if I want to use them, even if I decide to go be a hermit who never interacts with the outside world again.

I will concede a use case for hermits.

Or travel. Even in the developed world you can be without internet or slow internet. I have Gemma 4 E4B on my phone that can process audio, image, and text if I have need to.

Re: Inference Optimization for MiMo v2.5: Pushing Hybrid SWA Efficiency to the Limit

#36

Post by CuriouslyC »

mapontosevenths wrote: post #30
SwellJoe wrote: post #16

I've been saying efficiency is the next "frontier" in AI, at least for LLMs that people use daily. Companies have started to really balk at token costs from the major providers, and there's some evidence that the cheaper Chinese models are chipping away at Anthropic/OpenAI dominance from below (as cheaper Chinese products have done in other industries for many years). I continue to think that the vendor that figures…

I agree, but maybe for different reasons. I think Karpathy is right. We need models that reason, not models that memorize. Karpathy calls it a "Cognitive Core", and it's essentially a small model that learns to reason and look up the data it needs as opposed to a giant model that memorizes all the data in the world and tries to process large chunks of it all at once with every thought. I think it will be based on the…

The theorem you want to pay attention to is the no free lunch theorem. The important thing to understand there is that the larger models give you "free lunch" in the sense that you can approximate more different systems accurately at the cost of model size. If there was a Karpathy style universal solver, it wouldn't be very smart unless we scaled it up.

This isn't to say that there aren't a fair amount of wasted parameters in current LLMs, but then we already kinda knew that since you can quantize models down to 3-4 bits per weight often times with minimal loss.

Re: Inference Optimization for MiMo v2.5: Pushing Hybrid SWA Efficiency to the Limit

#37

Post by CuriouslyC »

TurdF3rguson wrote: post #17
trollbridge wrote: post #10

Earlier quoted context omitted.

Their game? Sell me tokens instead of me buying them from an American lab for a higher price. Publishing open weights gives me more confidence in the model, and ironically makes me less anxious about making sure I can replace the cloud usage with a local alternative. Whereas I’m very nervous right now with relying on 5.6-Sol - what if they triple the price, nerf it, etc.?

> Publishing open weights gives me more confidence in the model Why? It's not like you can audit weights like you can with code. > what if they triple the price, nerf it, etc.? What if an open weights infra provider does that? What's the difference?

You might not be able to audit the weights, but there are people with the skill set to do it.

If providers decide to jack the price, open weights lets you find a new provider without losing your fine tunes and having to re-do workflows, etc like you would if you switched off a frontier lab model.

Re: Inference Optimization for MiMo v2.5: Pushing Hybrid SWA Efficiency to the Limit

#38

Post by CuriouslyC »

yogthos wrote: post #12
LaurensBER wrote: post #2

It's really cool and interesting to see the kind of engineering that goes into Xiaomi (and Deepseeks) inference optimizations. Z.ai has also published some interesting papers although I haven't had a chance to go through them yet. It does inspire hope that the Chinese labs seem to be so open although the sceptic in me does wonder what their end game is. Surely, from a purely economic perspective it would be wiser to…

What Chinese firms are doing makes perfect sense from the commercial perspective actually because they understand how a classic commoditization spiral works. The reality is that models themselves are general commodities and there's just not enough difference between them. A company can get ahead of others by a few months, but then the rest quickly close the gap. It's a really low margin business because there's no wa…

The Chinese labs incentives is to run inference for the world, because inference can run on the homegrown Chinese chips (giving them a guaranteed market for their hardware) and they have cheap plentiful power.

The US frontier labs have an incentive to do deals with large firms to act like a contract research organization, taking royalties on creations/discoveries. Alex Karp called this out in his rant ("Why charge for tokens, take a %") and he's basically right about this.

Re: Inference Optimization for MiMo v2.5: Pushing Hybrid SWA Efficiency to the Limit

#39

Post by yorwba »

alightsoul wrote: post #15
LaurensBER wrote: post #2

It's really cool and interesting to see the kind of engineering that goes into Xiaomi (and Deepseeks) inference optimizations. Z.ai has also published some interesting papers although I haven't had a chance to go through them yet. It does inspire hope that the Chinese labs seem to be so open although the sceptic in me does wonder what their end game is. Surely, from a purely economic perspective it would be wiser to…

its a governement mandate that states that AI research must be open source ,that's one benefit of communism

There is no such mandate. ByteDance keeps their models closed. So does iFlyTek. Qwen Max is closed as well.

Re: Inference Optimization for MiMo v2.5: Pushing Hybrid SWA Efficiency to the Limit

#40

Post by perching_aix »

nmfisher wrote: post #28

Related, I was given access to mimo-v2.5-ultraspeed, which is amazing. This is now my expectation for speed, it’s fast enough for me to stay mentally engaged rather than getting distracted waiting for the agent to churn.

Is it the same quality as base mimo v2.5, or different? I've been enjoying regular mimo v2.5 quite a bit via opencode, if ultraspeed provides the same quality, that's crazy.
Post reply on HN