Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second

mimo.xiaomi.com

Post reply on HN
Print view View original

51–60 of 512 posts

Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second

#51

Post by maxdo »

eli wrote: post #38

Neat. The frontier models have gotten pretty impressive, but they're all a bit too slow for interactive, human-in-the-loop coding. It incentivizes vibecoding and running multiple agents in parallel. A fast agent feels more like a partner. For a while I was running Cerebras GLM 4.7 for a bunch of tasks. Not a very smart model, but it's fantastic to be have a live prototype of a site up and be able to type "make the fo…

i tried glm 4.7 for agents that write code. simple scripts 200-1000 LOC. extremely bad . Had to abandon cerebras oferning, their smart models are only on enterprise plan.

Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second

#52

Post by varispeed »

throwa356262 wrote: post #18
minraws wrote: post #6

Assuming they mean 8xA100 or similar, that's some rather insane performance, and at just 3x the cost, it still quite cheap-ish. With some optimisations this might be quite interesting. I think the margins are getting quite compressed with this one, since it isn't included in token plan and the actual costs increase are much higher than just 3x. But still fairly decent.

Suspect this will be included once out of beta but at a higher credit/token ratio. Remember, these guys are not VC backed. Anything they do must break even

Chinese "companies" are not companies in the western sense, but more like government departments with capitalist styling to deceive the western audience.

From that point of view, they have as much money as they need. That's why there is no "VC", because Chinese government assumes that role.

Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second

#53

Post by serf »

Accacin wrote: post #23
atemerev wrote: post #3

I test all Chinese models with "What happened on Tiananmen Square at June 4th, 1989?" prompt. MiMo-2.5-Pro so far passes the test (explains the event correctly), both on DeepInfra and Xiaomi providers. So not bad.

Can I ask an honest question? Why does that matter in the slightest? LLMs come out with completely incorrect information all the time, and Western LLMs are censored for various topics too. It's such a weird "Gotcha" that seems to only assume that Chinese LLMs might censor something.

>It's such a weird "Gotcha" that seems to only assume that Chinese LLMs might censor something.

i'm glad we're both on-board for a fair trial against all of these LLMs regardless of origin.

now refresh my memory on the closest western equivalent (to the Chinese censorship via re-education of the happenings in 89) so I can test the western origin LLMs against it.

Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second

#54

Post by Octoth0rpe »

qsera wrote: post #30

Tokens per seconds is the "Megapixels" of AI marketing!

I mean, sure, in the sense that they're a real and meaningful number for most of the spectrum on offer, and only gets silly when the number gets too high? There's a pretty big usability difference between 10t/s and 100t/s, and I can imagine similarly for 100->1000. I don't know about > 1000, but let's not pretend that the number is meaningless.

Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second

#55

Post by moffkalast »

vlovich123 wrote: post #14
moffkalast wrote: post #10

42B active params, sliding window attention. There's your tradeoff.

Sliding window for the draft model, not for the main. 42B for active params because it’s a sparse MoE which is a common technique for the larger models to not get bottlenecked by memory bandwidth.

Seems to be for both according to the spec [0], maybe it's wrong though.

128 sounds really tiny, I wonder if they mean some kind of blocks?

[0] https://huggingface.co/XiaomiMiMo/MiMo-V2.5-Pro-FP4-DFlash#4...

Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second

#56

Post by oulipo2 »

serpix wrote: post #16

I may sound like a shill, but exponential growth and all. We are going to get near instant software from prompt, multiple ones and then choose the best one. Discussions about choosing a library with the best syntactic sugar method naming is just as crazy as suggesting we type in assembly.

You won't. Because 80% of the complexity is just "knowing what to build". You will get something that gives you a prototype in 1 min, then you break it, then you get a slightly better prototype one one side, but newly broken in another way, and you're going to repeat over and over.

Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second

#57

Post by sagarp »

serpix wrote: post #16

I may sound like a shill, but exponential growth and all. We are going to get near instant software from prompt, multiple ones and then choose the best one. Discussions about choosing a library with the best syntactic sugar method naming is just as crazy as suggesting we type in assembly.

The models might be so fast that they can autocomplete your prompt before you even finish it, and generate dozens of possible applications before you're even done asking.

Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second

#58

Post by goyozi »

Fast AI seems genuinely exciting and somewhat unsettling to me. Right now Claude is faster than me on some tasks but we’re at least close. I have a prompt to clean up a PR that’s been running for 1h now and I expect it to take another few. It’s hard to imagine how the workflow would look like if it was near-instant. On the one hand, it might be easier to focus. Some prompts take so long that I start to multitask and regret it later. On the other, AI that takes a few seconds to max few minutes to solve what used to take hours or days? That’s a game changer and I don’t even know where we fit in.

Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second

#59

Post by h14h »

The gated "ultra-speed" phenomenon seen here and with the Cerebras Kimi K2.6 release, while understandable, is somewhat troubling IMO.

Getting ~1000 TPS on near-frontier intelligence is a step change, and enables whole new use-cases for applications. Seeing limited compute resources beget selective access makes me worry for the future of competition.

Re: MiMo-v2.5-Pro-UltraSpeed: 1T model with 1000 tokens per second

#60

Post by philipkglass »

harel wrote: post #36

A few things in life I can't fully grasp why they are so sought after. One is that constant need to exhibit growth. As if being massive and staying as massive is not good enough, one has to always and continuously grow. The other is constant speed increases. We're already operating at 50x speed. My output is much wider and so much faster, I am sometimes my own bottleneck. And now as if that is not enough we want more…

I remember when I had to wait minutes to get a high resolution image over a dialup connection. When computer and communications hardware advanced enough that I could get 30 high resolution images every second, there were brand new uses. In the case of LLMs, I could imagine that much faster operations allow you to introduce them as parts of systems that need to react to the real world at high speed, like factory equipment. Showing that a model can do the usual LLM tasks at extremely high speed is just a demo proving that the approach works.
Post reply on HN