Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Xiaomi MiMo Reasoning Model

github.com

Post reply on HN
Print view View original

191–200 of 203 posts

Re: Xiaomi MiMo Reasoning Model

#191

Post by eru »

loufe wrote: post #182
eru wrote: post #176

Earlier quoted context omitted.

That's wrong. Many LLMs are trained on synthetic data produced by other LLMs. (Indirectly, they may be trained on pirated books. Sure. But not directly.)

Likely the case for established model makers, but barring illegal use of outputs from other companies' models, a "first generation" model would still need this as a basis, no?

Why illegal? The more open models (or at least open-weight models) should allow using their outputs. Details depend on license.

But yes, 'first generation' models would be trained on human text almost by definition. My comment was only to contradict the claim that 'all LLMs' are trained from stolen text, by noting that some LLMs aren't trained (directly) on human text at all.

Re: Xiaomi MiMo Reasoning Model

#192

Post by overfeed »

yorwba wrote: post #172
overfeed wrote: post #126

Earlier quoted context omitted.

> That's clearly a different question. It'd be possible for these models to be Mandarin-first while still supporting other languages What would a hypothetical "Mandarin-first" model look like to you? I challenge the notion that the current models are "English-first" - that is an unsubstantiated opinion not supported by fact. I bet, dollars to donuts, these models are SoTA in Mandarin as well. When framed that way, as…

> What would a hypothetical "Mandarin-first" model look like to you? Given a language-agnostic prompt like "12 + 89", any explanatory text it outputs could be expected to be in Mandarin most of the time. According to this test, Xiaomi's MiMo-7B-RL is an English-first model.

"12 + 89" uses the latin alphabet and is in no way language-agnostic in this context. I expect borrowed constructs to appear relatively more frequently in the language they were borrowed from.

Now I'm curious how Mistral models would respond to a "language-agnostic" phrases like "Rendezvous" or "coup d'etat"

Re: Xiaomi MiMo Reasoning Model

#193

Post by yorwba »

overfeed wrote: post #192
yorwba wrote: post #172

Earlier quoted context omitted.

> What would a hypothetical "Mandarin-first" model look like to you? Given a language-agnostic prompt like "12 + 89", any explanatory text it outputs could be expected to be in Mandarin most of the time. According to this test, Xiaomi's MiMo-7B-RL is an English-first model.

"12 + 89" uses the latin alphabet and is in no way language-agnostic in this context. I expect borrowed constructs to appear relatively more frequently in the language they were borrowed from. Now I'm curious how Mistral models would respond to a "language-agnostic" phrases like "Rendezvous" or "coup d'etat"

You may think of these symbols as "Latin" because they're how people writing in Latin script happen to write mathematical expressions, but the exact same symbols are also used by Mandarin speakers, as well as in numerous other scripts. Writing math in Chinese characters is literally as uncommon as someone writing "twelve plus eighty-nine" in English.

In contrast, your examples would be spelled « rendez-vous » and « coup d’État » in French, i.e. easily distinguishable from their English descendants.

Re: Xiaomi MiMo Reasoning Model

#194

Post by syndeo »

olalonde wrote: post #108
throwaway519 wrote: post #95

Earlier quoted context omitted.

The pendulum already turned back. The current generation under 20 grew up with touchscreens. That obseletes input with pinyin; many don't care if the device has no keyboard.

> That obseletes input with pinyin Uh? Pinyin input is by far the most popular input technique in China. I rarely see anyone using handwriting input. That being said, it has nothing to do with English winning. It's just a Chinese input technique that uses the latin alphabet. English fluency in China is not very common, especially spoken English.

My father-in-law here in China uses handwriting input, but everyone else I've seen here uses Pinyin, and it's totally fast and natural for them.

And very true about the English. With some exceptions (of course), folks here maybe know a handful of words at best, and even then, pronunciation is usually pretty rough. People here really aren't using it; they are perfectly comfortable with their Chinese, and why wouldn't they be?

Anyone saying otherwise clearly hasn't been here to see it firsthand.

Re: Xiaomi MiMo Reasoning Model

#195

Post by syndeo »

34679 wrote: post #77
lvl155 wrote: post #65

Why are there so many English-first AI models from China? Are they not interested in serving their own population? Or is it that if they publish Chinese-first models it won't get publicity in the West?

Nearly everyone in the urban areas of China spoke some English when I visited way back in 1995. It's a bilingual society.

Not sure which part you were in, but this is just not true in my experience. I've been to Beijing, Shenzhen, Guangzhou, and some others, and Mandarin really is a must if you want to even have a chance of communicating. I can't imagine how I'd function here if I only had English.

I've not yet been to Shanghai, and while I would expect the English-speaking percentage to be a bit higher, it would still likely only be in the single-digits by my estimation.

Re: Xiaomi MiMo Reasoning Model

#196

Post by numpad0 »

Svoka wrote: post #143
div72 wrote: post #142

Earlier quoted context omitted.

> The largest portion of all languages in Common Crawl https://commoncrawl.github.io/cc-crawl-statistics/plots/lang...

Thanks! I wonder where this discrepancy comes from

My guess is that reference counting at depth=1 only capture non-$LANG content which text parts don't matter a lot, e.g. photo galleries.

Re: Xiaomi MiMo Reasoning Model

#197

Post by xpe »

paulluuk wrote: post #174
xpe wrote: post #155

The README says "RL" without specifying what kind of RL is used. Researchers: I know you are busy, and I know good writing takes time, but please don't skip this kind of detail.

I assume they mean "Reinforcement Learning", and it's been a decade since I studied AI in university, but isn't it perfectly valid to just say "RL"? What kind of specificity are you looking for, whether they used Q-Learning or some other algorithm?

I wouldn’t phrase it as a matter of “validity”. I would phrase it as a question of transparency.

Putting a model out in public without clearly explaining how it works doesn’t meet my bar for a proper scientific exchange of knowledge. Perhaps they are being intentionally vague for competitive reasons.

RL is a generic term that can be mixed and matched with various other methods. In the context of LLMs, often some variation of RLHF is used.

But the authors don’t even say “RLHF”, much less explain their methodology. Understanding this isn’t just a matter of academic interest; it has implications for understanding and using this work.

I’m often concerned by the writing quality of ML/AI papers but this strikes me as particularly disappointing.

It is increasingly important to have confidence that the creators of AI systems are thoughtful and thorough. I want to see their reasoning. I want to understand the trade-offs they make and why.

Re: Xiaomi MiMo Reasoning Model

#198

Post by pests »

SchemaLoad wrote: post #169
pests wrote: post #165

Earlier quoted context omitted.

Why did you choose Google when its classic lore they named it after a real word "googol" and the current spelling is just a typo from the first investor check.

It still doesn't really mean anything. Knowing the lore behind the name doesn't let you understand it any more than just taking it as a random name.

It's still based on a real word though.

Re: Xiaomi MiMo Reasoning Model

#199

Post by ii41 »

hnfong wrote: post #186
sidibe wrote: post #157

Earlier quoted context omitted.

It is. In Chinese speaking countries where there's google available, no one is using Baidu

There's only ONE* Chinese speaking country, at least if you only count those that have a Chinese speaking majority population, or uses Chinese as the official language. * for various interpretations of one.

Chinese is one of the offical languages of Singapore.

Re: Xiaomi MiMo Reasoning Model

#200

Post by ainch »

xpe wrote: post #155

The README says "RL" without specifying what kind of RL is used. Researchers: I know you are busy, and I know good writing takes time, but please don't skip this kind of detail.

The technical report does go into a lot of depth about how they use RL, such as the modified GRPO objective they use. As far as the README, I imagine most people active in the field understand the implications of "RL" for a reasoning model.
Post reply on HN