Live data from Hacker News

Xiaomi MiMo Reasoning Model

github.com

201–203 of 203 posts

Re: Xiaomi MiMo Reasoning Model

#201
post #197

Earlier quoted context omitted.

I assume they mean "Reinforcement Learning", and it's been a decade since I studied AI in university, but isn't it perfectly valid to just say "RL"? What kind of specificity are you looking for, whether they used Q-Learning or some other algorithm?

I wouldn’t phrase it as a matter of “validity”. I would phrase it as a question of transparency. Putting a model out in public without clearly explaining how it works doesn’t meet my bar for a proper scientific exchange of knowledge. Perhaps they are being intentionally vague for competitive reasons. RL is a generic term that can be mixed and matched with various other methods. In the context of LLMs, often some vari…

If you put it like that, I absolutely agree with you, except that I suppose I don't really consider this an exchange of knowledge but more like the release of an open-source project: the only thing they need to publish are instructions on how to use it. I don't think they’re really interested in anyone improving their model by themselves or reproducing the work. It would be amazing if they did, though!

Re: Xiaomi MiMo Reasoning Model

#202
post #193

Earlier quoted context omitted.

"12 + 89" uses the latin alphabet and is in no way language-agnostic in this context. I expect borrowed constructs to appear relatively more frequently in the language they were borrowed from. Now I'm curious how Mistral models would respond to a "language-agnostic" phrases like "Rendezvous" or "coup d'etat"

You may think of these symbols as "Latin" because they're how people writing in Latin script happen to write mathematical expressions, but the exact same symbols are also used by Mandarin speakers, as well as in numerous other scripts. Writing math in Chinese characters is literally as uncommon as someone writing "twelve plus eighty-nine" in English. In contrast, your examples would be spelled « rendez-vous » and « c…

> You may think of these symbols as "Latin" because they're how people writing in Latin script happen to write mathematical expressions

No need for scare-quotes, Latin script is a proper noun and a technical term with precise meaning wrt text encoding - not "what I think."

> the exact same symbols are also used by Mandarin speakers, as well as in numerous other scripts. Writing math in Chinese

Which unicode code points do the Mandarin speakers and "numerous other scripts" use to write "12 + 89"? Could it be the very same code points as Latin script, which then are tokenized to the same vectors that the LLMs learn to associate more with English text rather than CJK in the latent space?

> i.e. easily distinguishable from their English descendants.

You're making broad assumptions about the tokenization design here that do not apply universally.

Re: Xiaomi MiMo Reasoning Model

#203
post #193

Earlier quoted context omitted.

You may think of these symbols as "Latin" because they're how people writing in Latin script happen to write mathematical expressions, but the exact same symbols are also used by Mandarin speakers, as well as in numerous other scripts. Writing math in Chinese characters is literally as uncommon as someone writing "twelve plus eighty-nine" in English. In contrast, your examples would be spelled « rendez-vous » and « c…

> You may think of these symbols as "Latin" because they're how people writing in Latin script happen to write mathematical expressions No need for scare-quotes, Latin script is a proper noun and a technical term with precise meaning wrt text encoding - not "what I think." > the exact same symbols are also used by Mandarin speakers, as well as in numerous other scripts. Writing math in Chinese Which unicode code poin…

Precisely because the exact same codepoints are used for digits and mathematical symbols, there's nothing script-specific about them and their linguistic association is determined by the training data mixture. A model trained predominantly on text scraped from Chinese websites would learn to associate them more with Mandarin than English in the latent space, since that would be the context where they most often appear.
Post reply on HN