Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Xiaomi MiMo Reasoning Model

github.com

Post reply on HN
Print view View original

151–160 of 203 posts

Re: Xiaomi MiMo Reasoning Model

#151

Post by novaRom »

rahimnathwani wrote: post #140
Havoc wrote: post #136

Earlier quoted context omitted.

One of the core design goals Georgi Gerganov had with GGUF was to not need other files. It's literally bullet point #1 in the specs >Single-file deployment >Full information: all information needed to load a model is contained in the model file, and no additional information needs to be provided by the user. https://github.com/ggml-org/ggml/blob/master/docs/gguf.md We literally just got rid of that multi file chaos o…

Most of the parameters you would include in ollama's ModelFile are things you would pass to llama.cpp using command line flags: https://github.com/ggml-org/llama.cpp/blob/master/examples/m... If you only ever have one set of configuration parameters per model (same temp, top_p, system prompt...), then I guess you can put them in a gguf file (as the format is extensible). But what if you want two different sets? You s…

This is why we use xdelta3, rdiff, and git

Re: Xiaomi MiMo Reasoning Model

#152

Post by tough »

Svoka wrote: post #143
div72 wrote: post #142

Earlier quoted context omitted.

> The largest portion of all languages in Common Crawl https://commoncrawl.github.io/cc-crawl-statistics/plots/lang...

Thanks! I wonder where this discrepancy comes from

probably under-indexing of non-english sources by these crawlers.

would be interesting if yandex opened some data sets!

Re: Xiaomi MiMo Reasoning Model

#153

Post by Havoc »

Been testing it a bit and overall pretty solid. The lengthy think times means one waits quite a while though. Longer than much larger models like say the recent qwen moe

That moe strikes me as the better overall tradeoff

Re: Xiaomi MiMo Reasoning Model

#154

Post by -__---____-ZXyw »

Jarwain wrote: post #148
-__---____-ZXyw wrote: post #133

Earlier quoted context omitted.

Source? This smacks of "I saw a headline once"-itis. Especially the fact that you refer to the Chinese characters as "calligraphy characters", as if that were the general term or something.

These are probably the headlines they're thinking about, https://www.globaltimes.cn/content/747853.shtml https://www.bbc.com/news/blogs-china-blog-28599392 Or more recently this one about character amnesia https://globalchinapulse.net/character-amnesia-in-china/ None of these really mean that English has won, though. Rather that phonetics-based writing systems are easier to remember and use, especially in conjunction…

Ah no, I know myself that there have been headlines here and there.

I'm pretty sure there was some controvery in the linguistic blogging community even at some stage over the last couple of years, with someone writing an essay claiming the Chinese character system was in some sense less advanced and maybe on the way out, and this leading to a serious response or two, the usual fiery academic affair. I can't locate it this instant though.

I moreso meant for OP's low-effort dramatisation to not go unanswered. Framing it as "winning" some sort of language battle is particularly silly.

Your musings are interesting though, and the topic certainly is a fascinating one. Languages that use morphemes for writing are wild. Symbology is a cool word also - surely there has to be a lisp blog somewhere with that word in the title.

Re: Xiaomi MiMo Reasoning Model

#155

Post by xpe »

The README says "RL" without specifying what kind of RL is used. Researchers: I know you are busy, and I know good writing takes time, but please don't skip this kind of detail.

Re: Xiaomi MiMo Reasoning Model

#156

Post by AlexCoventry »

yyhhsj0521 wrote: post #114
lvl155 wrote: post #65

Why are there so many English-first AI models from China? Are they not interested in serving their own population? Or is it that if they publish Chinese-first models it won't get publicity in the West?

Chinese internet mostly consists of a few closed gardens tightly controlled by big corps. Crawlers simply don't work when each company employs an army of engineers to guard their data. Many of the most popular websites are also app only. It's impossible to get the corpus necessary to train a good LLM.

DeepSeek claims they had 12% more Chinese tokens than English, in their training corpus for DeepSeek V2, FWIW.

https://arxiv.org/pdf/2405.04434#page=12

> Our tokenized pretraining corpus contains 8.1T tokens, where Chinese tokens are approximately 12% more than English ones.

Re: Xiaomi MiMo Reasoning Model

#157

Post by sidibe »

littlestymaar wrote: post #150
choutianxius wrote: post #80

Earlier quoted context omitted.

One reason is that there is no "good" search engine in China. The most popular one, Baidu, is like garbage compared to Google search. The most useful training data in Chinese would likely be from the social media and video sharing platforms, which I guess is much more difficult to crawl and clean up.

> The most popular one, Baidu, is like garbage compared to Google search It must be very bad when you see the walking turd that Google search has become over the years…

It is. In Chinese speaking countries where there's google available, no one is using Baidu

Re: Xiaomi MiMo Reasoning Model

#158

Post by xpe »

otabdeveloper4 wrote: post #90
ramesh31 wrote: post #5

These benchmark numbers cannot be real for a 7b model

LLM benchmarks are mostly bullshit right now. Wait a few years until the hype cycle returns to sanity.

>> These benchmark numbers cannot be real for a 7b model

> LLM benchmarks are mostly bullshit right now. Wait a few years until the hype cycle returns to sanity.

This could mean a lot of things. Can you be a bit more specific? It's one thing to say benchmarks are gamed. Another to say models end up being trained on the benchmark indirectly. Another to say they the particular experimental setup during the benchmark is unclear. Another to say mapping a benchmark to a real use case is hard. Are you saying some/all of these claims?

Have you plotted MiMo versus others? Another comment suggests smaller models are performing better than expected. Any comment on that?

Re: Xiaomi MiMo Reasoning Model

#159

Post by badmonster »

MiMo-7B claims to outperform larger models like Qwen-32B and match OpenAI o1-mini on math/code benchmarks — all with a 7B model trained from scratch. Is this a sign that pretraining + RLHF optimization is finally outpacing scale? Or are we just getting better at benchmarking narrow capabilities?

Re: Xiaomi MiMo Reasoning Model

#160

Post by Barrin92 »

lvl155 wrote: post #65

Why are there so many English-first AI models from China? Are they not interested in serving their own population? Or is it that if they publish Chinese-first models it won't get publicity in the West?

>Or is it that if they publish Chinese-first models it won't get publicity in the West?

This is a large part of it. Kai-Fu Lee's company (https://www.01.ai/) has been publishing open source Chinese language/market focused models pretty early, but the entire conversation around Chinese tech just isn't available to you if you don't speak Chinese, in particular these days given that good English language reporting on the Chinese tech sector just seems very scarce.

Post reply on HN