Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Omnilingual ASR: Advancing automatic speech recognition for 1600 languages

ai.meta.com

Post reply on HN
Print view View original

31–40 of 48 posts

Re: Omnilingual ASR: Advancing automatic speech recognition for 1600 languages

#31

Post by sipjca »

dSebastien wrote: post #25

I'm going to test this with Voice AI to see how it works compared to Whisper and Parakeet https://voice-ai.knowii.net

looks like a paid and closed source fork of the free and open source project Handy: https://github.com/cjpais/Handy

can't say for sure, but a lot of the UI (and text) is quite familiar. the history page is a near rip off which is a giveaway.

i believe the mit license should be distributed since it's almost certainly a derivative work.

"The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software."

I can't confirm if the license is infact distributed, since I would have to pay $50, which quite frankly I'm not going to do.

a bit sad to see a ui reskin claimed as original work. the reskin is totally fine, but I believe the license must be distributed. i believe in the proliferation of this software so im happy to see this overall (it's good enough someone wants to charge for it! that's a big win!) but it's just a bit of a shame how this project has gone about it imo.

Re: Omnilingual ASR: Advancing automatic speech recognition for 1600 languages

#32

Post by District5524 »

I agree that this is a very exciting and really crucial research and I'm glad there is funding for this. But it's very strange that Hungarian is marked as "highly endangered" at https://aidemos.atmeta.com/omnilingualasr/language-globe Highly endangered is supposed to mean "The language is used by grandparents and older generations; while the parent generation may still understand the language, they typically do not speak it to children or among themselves." Then why is Hungarian marked as such? Obviously not true with 14 million active speakers and being the 20th in terms of the most language resources published on the Internet. Additionally, the feedback mechanism seems also broken ("There was an error submitting your feedback. Please try again.")

Re: Omnilingual ASR: Advancing automatic speech recognition for 1600 languages

#33

Post by District5524 »

tmikaeld wrote: post #26

Swedish Status: Endangered "The child-bearing generation can use the language among themselves, but it is seldom being transmitted to children." What!? A lot must have changed in one generation..

Yes, there seems to be lots of mistakes and no easy way to mark it. Highly endangered: Malayalam (=35 million speakers), Hungarian (14 million), Uighur (11 million), or Swedish as endangered... These are quite obvious mistakes even for a layperson.

Re: Omnilingual ASR: Advancing automatic speech recognition for 1600 languages

#34

Post by internet_points »

ks2048 wrote: post #19

Just killed my startup. https://6k.ai Half joking - hopefully, we can still contribute something to this to this field. Looking forward to doing some tests with this.

what is the "Penguin" language?

Also, 1.6k < 6k, and I highly doubt this model is anywhere near as good as it is on EU languages for most of them.

Re: Omnilingual ASR: Advancing automatic speech recognition for 1600 languages

#35

Post by internet_points »

District5524 wrote: post #32

I agree that this is a very exciting and really crucial research and I'm glad there is funding for this. But it's very strange that Hungarian is marked as "highly endangered" at https://aidemos.atmeta.com/omnilingualasr/language-globe Highly endangered is supposed to mean "The language is used by grandparents and older generations; while the parent generation may still understand the language, they typically do not s…

Finnish: "safe" – sounds right

South Estonian: "vulnerable" – sure, yeah

Karelian: "endangered" – seems correct

Swedish: also "endangered" – wat

Ghari (12k speakers): "safe" – :facepalm:

Are these really language-vulnerability ratings or did they just make a mapping from Trump's tariff rates?

Re: Omnilingual ASR: Advancing automatic speech recognition for 1600 languages

#36

Post by District5524 »

internet_points wrote: post #35
District5524 wrote: post #32

I agree that this is a very exciting and really crucial research and I'm glad there is funding for this. But it's very strange that Hungarian is marked as "highly endangered" at https://aidemos.atmeta.com/omnilingualasr/language-globe Highly endangered is supposed to mean "The language is used by grandparents and older generations; while the parent generation may still understand the language, they typically do not s…

Finnish: "safe" – sounds right South Estonian: "vulnerable" – sure, yeah Karelian: "endangered" – seems correct Swedish: also "endangered" – wat Ghari (12k speakers): "safe" – :facepalm: Are these really language-vulnerability ratings or did they just make a mapping from Trump's tariff rates?

My new favourite mistake is Malayalam being highly endangered...

Re: Omnilingual ASR: Advancing automatic speech recognition for 1600 languages

#37

Post by akreal »

____tom____ wrote: post #23

What I really want to know is how well these could work for non-human languages. No, not aliens, but chimpanzees, dolphins, bonobos. We have hundreds or thousands of hours of recordings. What would it take to start working on them?

There is a dolphin language model project from Google and Georgia Tech: https://blog.google/technology/ai/dolphingemma/

Re: Omnilingual ASR: Advancing automatic speech recognition for 1600 languages

#38

Post by copypirate »

sipjca wrote: post #31
dSebastien wrote: post #25

I'm going to test this with Voice AI to see how it works compared to Whisper and Parakeet https://voice-ai.knowii.net

looks like a paid and closed source fork of the free and open source project Handy: https://github.com/cjpais/Handy can't say for sure, but a lot of the UI (and text) is quite familiar. the history page is a near rip off which is a giveaway. i believe the mit license should be distributed since it's almost certainly a derivative work. "The above copyright notice and this permission notice shall be included in all cop…

I thought it looked familiar! Looks like they only changed some of the UI/colors lol.

Re: Omnilingual ASR: Advancing automatic speech recognition for 1600 languages

#39

Post by yorwba »

internet_points wrote: post #35
District5524 wrote: post #32

I agree that this is a very exciting and really crucial research and I'm glad there is funding for this. But it's very strange that Hungarian is marked as "highly endangered" at https://aidemos.atmeta.com/omnilingualasr/language-globe Highly endangered is supposed to mean "The language is used by grandparents and older generations; while the parent generation may still understand the language, they typically do not s…

Finnish: "safe" – sounds right South Estonian: "vulnerable" – sure, yeah Karelian: "endangered" – seems correct Swedish: also "endangered" – wat Ghari (12k speakers): "safe" – :facepalm: Are these really language-vulnerability ratings or did they just make a mapping from Trump's tariff rates?

The Ethnologue link in footnote 7 of the paper has utm_source=chatgpt.com at the end, so I suspect whoever was tasked with listing languages and determining their status thought this wasn't important enough to do it themselves and just had ChatGPT give them a list. FWIW, Ethnologue does say that Ghari is "Stable" https://www.ethnologue.com/language/gri/ Meanwhile Swedish is "Institutional," the highest possible level of vitality https://www.ethnologue.com/language/swe/

Re: Omnilingual ASR: Advancing automatic speech recognition for 1600 languages

#40

Post by yorwba »

nshm wrote: post #22
lostmsu wrote: post #12

Earlier quoted context omitted.

Not sure, I recorded 3 seconds of voice (a single sentence) and the hf demo misrecognized about half of the words.

And moreover, you can not tune those models for practical applications. The model is originally trained on very clean data, so lower layers are also not very stable for diverse inputs. To finetune you have to update the whole model, not just upper layers.

In section 5.7.5, they fine-tune for "11 low-resource languages, with between 5-10 hours of training data and at least 1 hour of validation splits." "CTC fine-tuning takes ≈1 hour of walltime on 32 GPUs for the 300M scale." If that's too expensive, you also have the option of supplying additional context for the LLM-based model (section 5.5).

As for "very clean data," see section 5.7.4: "Omnilingual + OMSF ASR was intentionally curated to represent naturalistic (i.e., often noisy) audio conditions, diverse speaker identities, and spontaneous, expressive speech."

Post reply on HN