Earlier quoted context omitted.
I assume they mean "Reinforcement Learning", and it's been a decade since I studied AI in university, but isn't it perfectly valid to just say "RL"? What kind of specificity are you looking for, whether they used Q-Learning or some other algorithm?
I wouldn’t phrase it as a matter of “validity”. I would phrase it as a question of transparency. Putting a model out in public without clearly explaining how it works doesn’t meet my bar for a proper scientific exchange of knowledge. Perhaps they are being intentionally vague for competitive reasons. RL is a generic term that can be mixed and matched with various other methods. In the context of LLMs, often some vari…
Xiaomi MiMo Reasoning Model
201–203 of 203 posts
Re: Xiaomi MiMo Reasoning Model
#202Earlier quoted context omitted.
"12 + 89" uses the latin alphabet and is in no way language-agnostic in this context. I expect borrowed constructs to appear relatively more frequently in the language they were borrowed from. Now I'm curious how Mistral models would respond to a "language-agnostic" phrases like "Rendezvous" or "coup d'etat"
You may think of these symbols as "Latin" because they're how people writing in Latin script happen to write mathematical expressions, but the exact same symbols are also used by Mandarin speakers, as well as in numerous other scripts. Writing math in Chinese characters is literally as uncommon as someone writing "twelve plus eighty-nine" in English. In contrast, your examples would be spelled « rendez-vous » and « c…
No need for scare-quotes, Latin script is a proper noun and a technical term with precise meaning wrt text encoding - not "what I think."
> the exact same symbols are also used by Mandarin speakers, as well as in numerous other scripts. Writing math in Chinese
Which unicode code points do the Mandarin speakers and "numerous other scripts" use to write "12 + 89"? Could it be the very same code points as Latin script, which then are tokenized to the same vectors that the LLMs learn to associate more with English text rather than CJK in the latent space?
> i.e. easily distinguishable from their English descendants.
You're making broad assumptions about the tokenization design here that do not apply universally.
Re: Xiaomi MiMo Reasoning Model
#203Earlier quoted context omitted.
You may think of these symbols as "Latin" because they're how people writing in Latin script happen to write mathematical expressions, but the exact same symbols are also used by Mandarin speakers, as well as in numerous other scripts. Writing math in Chinese characters is literally as uncommon as someone writing "twelve plus eighty-nine" in English. In contrast, your examples would be spelled « rendez-vous » and « c…
> You may think of these symbols as "Latin" because they're how people writing in Latin script happen to write mathematical expressions No need for scare-quotes, Latin script is a proper noun and a technical term with precise meaning wrt text encoding - not "what I think." > the exact same symbols are also used by Mandarin speakers, as well as in numerous other scripts. Writing math in Chinese Which unicode code poin…