Earlier quoted context omitted.
That's a fair point. I think GPU clusters are the big one, the rest sounds like a good fit for volunteer work.
Or sharing GPU compute. Crowd sourcing.
Ooooh I can see a Seti@Home setup working
Xiaomi MiMo Reasoning Model
121–130 of 203 posts
Re: Xiaomi MiMo Reasoning Model
#122Earlier quoted context omitted.
Or sharing GPU compute. Crowd sourcing.
Ooooh I can see a Seti@Home setup working
Easier said than done, training is usually done on "big iron" GPUs which are a cut above any hardware that consumers have lying around, and the clusters run on multi-hundred-gigabit networks. Even if you scaled it down to run on gaming cards, and gathered enough volunteers, the low bandwidth and high latency of the internet would still be a problem.
Re: Xiaomi MiMo Reasoning Model
#123Why are there so many English-first AI models from China? Are they not interested in serving their own population? Or is it that if they publish Chinese-first models it won't get publicity in the West?
Why are so many American models multi-lingual, supporting hundreds of languages not commonly spoken in the United States?
Could it be that being multilingual results in a larger pool of human knowledge on the technical side compared to training on just a single language or 2. And on the business side, supporting more languages results in a larger TAM (total addressable market). Using english-language dataset for training LLMs is the default, not the other way like you insinuate.
Re: Xiaomi MiMo Reasoning Model
#124Why are there so many English-first AI models from China? Are they not interested in serving their own population? Or is it that if they publish Chinese-first models it won't get publicity in the West?
Why are so many American models multi-lingual, supporting hundreds of languages not commonly spoken in the United States? Could it be that being multilingual results in a larger pool of human knowledge on the technical side compared to training on just a single language or 2. And on the business side, supporting more languages results in a larger TAM (total addressable market). Using english-language dataset for trai…
That's clearly a different question. It'd be possible for these models to be Mandarin-first while still supporting other languages, like American models are English-first while doing the same, but that's not what's happening.
Re: Xiaomi MiMo Reasoning Model
#125Earlier quoted context omitted.
Last time I did that I was also impressed, for a start. Problem was that of a top ten book recommendations only the first 3 existed and the rest was a casually blended hallucination delivered in perfect English without skipping a beat. "You like magic? Try reading the Harlew Porthouse series by JRR Marrow, following the orphan magicians adventures in Hogwesteros" And the further towards the context limit it goes the…
LLMs are not search engines…
Many tasks that one might want to give a model end up implicitly including search as a subtask. For example, "plan me a trip to Santiago" obviously requires the model to understand details about the real city of Santiago. Less obviously, "write me a Python script to do ..." requires they understand APIs, libraries, etc., the same things you might ask a search engine to pull up. The tasks which do not require a coherent + mostly-correct exterior-world-model are relatively few -- text processing (e.g. "proofread this") is a big one; calculation tasks fit, but LLMs are also bad at those.
Re: Xiaomi MiMo Reasoning Model
#126Earlier quoted context omitted.
Why are so many American models multi-lingual, supporting hundreds of languages not commonly spoken in the United States? Could it be that being multilingual results in a larger pool of human knowledge on the technical side compared to training on just a single language or 2. And on the business side, supporting more languages results in a larger TAM (total addressable market). Using english-language dataset for trai…
That's clearly a different question. It'd be possible for these models to be Mandarin-first while still supporting other languages, like American models are English-first while doing the same, but that's not what's happening.
> That's clearly a different question. It'd be possible for these models to be Mandarin-first while still supporting other languages
What would a hypothetical "Mandarin-first" model look like to you?
I challenge the notion that the current models are "English-first" - that is an unsubstantiated opinion not supported by fact. I bet, dollars to donuts, these models are SoTA in Mandarin as well. When framed that way, asking "Why are they marketed as English-speaking models outside of China" or "Why are they really good at English" are simply not interesting questions - they have obvious answers.
Re: Xiaomi MiMo Reasoning Model
#127Why are there so many English-first AI models from China? Are they not interested in serving their own population? Or is it that if they publish Chinese-first models it won't get publicity in the West?
I was under the impression that we just see the English stuff given that we're using English news channels.
Re: Xiaomi MiMo Reasoning Model
#128Re: Xiaomi MiMo Reasoning Model
#129Why are there so many English-first AI models from China? Are they not interested in serving their own population? Or is it that if they publish Chinese-first models it won't get publicity in the West?
One reason is that there is no "good" search engine in China. The most popular one, Baidu, is like garbage compared to Google search. The most useful training data in Chinese would likely be from the social media and video sharing platforms, which I guess is much more difficult to crawl and clean up.
A few thousand years of literature ain’t nothing…
Re: Xiaomi MiMo Reasoning Model
#130Why are there so many English-first AI models from China? Are they not interested in serving their own population? Or is it that if they publish Chinese-first models it won't get publicity in the West?
CommonCrawl [1] is the biggest and most easily accessible legally acquired crawling dataset around, collecting data since 2008. Pretty much everyone uses this as their base dataset for training foundation LLMs and since it's mostly English, all models perform well in English.