Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta AI announces Massive Multilingual Speech code, models for 1000+ languages

github.com

Post reply on HN
Print view View original

131–140 of 232 posts

Re: Meta AI announces Massive Multilingual Speech code, models for 1000+ languages

#131

Post by omneity »

I was super excited at this, but digging through the release [0] one can see the following [1]. While using Bible translations is indeed better than nothing, I don't think the stylistic choices in the Bible are representative of how people actually speak the language, in any of the languages I can speak (i.e. that I am able to evaluate personally).

Religious recordings tend to be liturgical, so even the pronunciation might be different to the everyday language. They do address something related, although more from a vocabulary perspective to my understanding [2].

So one of their stated goals, to enable people to talk to AI in their preferred language [3], might be closer but certainly a stretch to achieve with their chosen dataset.

[0]: https://about.fb.com/news/2023/05/ai-massively-multilingual-...

[1]: > These translations have publicly available audio recordings of people reading these texts in different languages. As part of the MMS project, we created a dataset of readings of the New Testament in more than 1,100 languages, which provided on average 32 hours of data per language. By considering unlabeled recordings of various other Christian religious readings, we increased the number of languages available to more than 4,000. While this data is from a specific domain and is often read by male speakers, our analysis shows that our models perform equally well for male and female voices. And while the content of the audio recordings is religious, our analysis shows that this doesn’t bias the model to produce more religious language.

[2]: > And while the content of the audio recordings is religious, our analysis shows that this doesn’t bias the model to produce more religious language.

[3]: > This kind of technology could be used for VR and AR applications in a person’s preferred language and that can understand everyone’s voice.

Re: Meta AI announces Massive Multilingual Speech code, models for 1000+ languages

#132

Post by generalizations »

mach1ne wrote: post #23
generalizations wrote: post #6

Earlier quoted context omitted.

Based on the availability of STT, TTS, and translation models available to download in that github repo, a real-life babelfish is 'only' some glue code away. Wild times we're living in.

Nah, full-on babelfish is simply not possible. The meaning of the beginning of a sentence can be modified retroactively by the end of the sentence. This means that the Babelfish must either be greatly delayed or awkwardly correct itself every once in a while.

So just add 5 seconds of latency?

Re: Meta AI announces Massive Multilingual Speech code, models for 1000+ languages

#133

Post by o1y32 »

tmpz22 wrote: post #83
crazygringo wrote: post #56

Earlier quoted context omitted.

They're intended for researchers/professionals not consumers, and I'm not sure how a video is going to be helpful? And the issue with a live demo is that these are resource-intensive, they're not just webpages. It's an entire project to figure out how to host them, scale them to handle peaks, pay for them, implement rate-limiting, and so forth. For the intended audience, download-and-run-it doesn't seem like an issue…

Its so weird to me that they'd do 99% of the effort and just skip past the 1% of work to provide a dumbed-down summary and instructions for broader appeal. Clearly these are released in part for public-relations and industry clout. Don't get me wrong they published this, it took a ton of work, they didn't have to do it. But its ultimately a form of gatekeeping that seems to come straight out of Academia. And honestly…

It's not 1% of the work. If you have ever written public documentation, or half-decent internal documentation that is understandable to people outside your team, you would know that it takes a lot of effort to get it right. Let alone something highly technical like this. If you get things wrong or skip things you think are trivial but not to others, you get more questions. And it is not a one-time thing, it is a living document that needs to be updated and revised over time.

Re: Meta AI announces Massive Multilingual Speech code, models for 1000+ languages

#134

Post by crazygringo »

egberts1 wrote: post #127
crazygringo wrote: post #98

Earlier quoted context omitted.

I stand corrected, thanks. There's a lot of info on the internet that says a written form of ASL doesn't exist, which is what I found when I Googled it. Looking into it, it seems very much at the experimental stage in terms of digital representation -- while Unicode symbols exist, they require being placed in 2D boxes (using a drawing tool like SVG). It seems like it's only in the past few years that there have been…

SignWriting has been documented at National Deafness section in California State University of Northridge South Library since 1968 and at Gallaudet University, Washington, D.C. since 1950s.

That's great, but it doesn't have anything to do with the points I raised.

Which is that I can't find any indication of a linear digital encoding that has been used to any appreciable extent that Meta could train on a corpus of it.

Which is why I'm struggling to understand why you're criticizing Meta? How could they realistically train a linear text model on ASL when the necessary content doesn't appear to exist?

Re: Meta AI announces Massive Multilingual Speech code, models for 1000+ languages

#135

Post by generalizations »

omneity wrote: post #131

I was super excited at this, but digging through the release [0] one can see the following [1]. While using Bible translations is indeed better than nothing, I don't think the stylistic choices in the Bible are representative of how people actually speak the language, in any of the languages I can speak (i.e. that I am able to evaluate personally). Religious recordings tend to be liturgical, so even the pronunciation…

This paragraph justifies the decision, IMHO.

> Collecting audio data for thousands of languages was our first challenge because the largest existing speech datasets cover 100 languages at most. To overcome this, we turned to religious texts, such as the Bible, that have been translated in many different languages and whose translations have been widely studied for text-based language translation research.

I think the choice was just between having any data at all, or not being able to support that language.

Re: Meta AI announces Massive Multilingual Speech code, models for 1000+ languages

#136

Post by coffeebeqn »

reaperman wrote: post #35
mach1ne wrote: post #23

Earlier quoted context omitted.

Nah, full-on babelfish is simply not possible. The meaning of the beginning of a sentence can be modified retroactively by the end of the sentence. This means that the Babelfish must either be greatly delayed or awkwardly correct itself every once in a while.

Close enough for practical value. Yes, big downside. But personally I’d use it.

I wonder how well a LLM could guess the end of the sentence though. Most of the time decently well?

Re: Meta AI announces Massive Multilingual Speech code, models for 1000+ languages

#137

Post by omneity »

generalizations wrote: post #135
omneity wrote: post #131

I was super excited at this, but digging through the release [0] one can see the following [1]. While using Bible translations is indeed better than nothing, I don't think the stylistic choices in the Bible are representative of how people actually speak the language, in any of the languages I can speak (i.e. that I am able to evaluate personally). Religious recordings tend to be liturgical, so even the pronunciation…

This paragraph justifies the decision, IMHO. > Collecting audio data for thousands of languages was our first challenge because the largest existing speech datasets cover 100 languages at most. To overcome this, we turned to religious texts, such as the Bible, that have been translated in many different languages and whose translations have been widely studied for text-based language translation research. I think the…

Of course, my critique is less "This project shouldn't exist", but rather "It seems to me there are several biases that affect the performance of this project in the context it was presented in".

This is a great project and an important stepping stone in a multilingual AI future.

Re: Meta AI announces Massive Multilingual Speech code, models for 1000+ languages

#139

Post by pmoriarty »

mach1ne wrote: post #23
generalizations wrote: post #6

Earlier quoted context omitted.

Based on the availability of STT, TTS, and translation models available to download in that github repo, a real-life babelfish is 'only' some glue code away. Wild times we're living in.

Nah, full-on babelfish is simply not possible. The meaning of the beginning of a sentence can be modified retroactively by the end of the sentence. This means that the Babelfish must either be greatly delayed or awkwardly correct itself every once in a while.

sometimes humans too don't know what they're going to say until they've said it

Re: Meta AI announces Massive Multilingual Speech code, models for 1000+ languages

#140

Post by bigdict »

qwertox wrote: post #68

I would like to use stuff like this as a side-project. Buy a Nvidia Geforce GPU and stick it into my 24/7 server and play around with it in my free time, to see what can be done. The issue with all these AI models is that there's no information on which GPU is enough for which task. I'm absolutely clueless if a single RTX 4000 SFF with its 20GB VRAM and only 70W of max power usage will be a waste of money, or really…

A 3080 Ti is sufficient to run the largest Whisper model (just under 12 GB VRAM).
Post reply on HN