I don't understand, given all they say, why this would not be made available to everyone at once? Why the limited release? They should have no trouble scaling it if it runs on a single rack.
MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second
11–20 of 512 posts
Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second
#12[flagged]
Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second
#13Given that MiMo is as cheap as Deepseek ( previous discussion: https://news.ycombinator.com/item?id=48282814 ) multiplying that by 3x for ultra speed is still shockingly cheap.
Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second
#1442B active params, sliding window attention. There's your tradeoff.
Sliding window for the draft model, not for the main. 42B for active params because it’s a sparse MoE which is a common technique for the larger models to not get bottlenecked by memory bandwidth.
Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second
#15I test all Chinese models with "What happened on Tiananmen Square at June 4th, 1989?" prompt. MiMo-2.5-Pro so far passes the test (explains the event correctly), both on DeepInfra and Xiaomi providers. So not bad.
No idea why you've been downvoted. This is excellent news.
Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second
#16I may sound like a shill, but exponential growth and all. We are going to get near instant software from prompt, multiple ones and then choose the best one.
Discussions about choosing a library with the best syntactic sugar method naming is just as crazy as suggesting we type in assembly.
Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second
#17These price and speed optimization from Chinese providers, combined with the raising prices from American ones will change the game sooner than later. Many companies are finding issues with the AI bills already.
Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second
#18Assuming they mean 8xA100 or similar, that's some rather insane performance, and at just 3x the cost, it still quite cheap-ish. With some optimisations this might be quite interesting. I think the margins are getting quite compressed with this one, since it isn't included in token plan and the actual costs increase are much higher than just 3x. But still fairly decent.
Suspect this will be included once out of beta but at a higher credit/token ratio.
Remember, these guys are not VC backed. Anything they do must break even
Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second
#19Cerebras is trialing Kimi K2.6 at 3000t/s (invite only). I'm excited for when the fast hardware gets more mainstream for frontier models. Models designed for speed on Nvidia are nice addition that could bridge the gap.
Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second
#20If MiMo v2.5 Pro can run at >1000tk/s on GPUs then I will soon expect the same from OpenAI/Anthropic/Google.