Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No Language Left Behind

ai.facebook.com

Post reply on HN
Print view View original

71–80 of 166 posts

Re: No Language Left Behind

#71

Post by bobsmooth »

Etheryte wrote: post #24

I'll believe it when I actually see it. I'm a native of a reasonably small language spoken by about a million people and never have I ever seen a good automatic translation for it. The only translations that are good are the ones that have been manually entered, and those that match the structure of the manually entered ones. I think the sentiment is laudable and wish godspeed to the people working on this, but for t…

There's a section where you can try reading translated children's books. See if your language is supported and how good the translation is.

Re: No Language Left Behind

#72

Post by bobsmooth »

daniel-cussen wrote: post #68
jw4ng wrote: post #41

Earlier quoted context omitted.

We think it's important for AI to truly support everyone in the world. A world where AI only serves a subset of the population is not ideal. In machine translation, this means supporting as many language as possible at high quality. We also imagine a future where anyone will be able to communicate with anyone else seamlessly; this also means solving translations for all languages.

Wouldn't that also entail a bot speaking in any language?

Text to speech is a separate problem.

Re: No Language Left Behind

#73

Post by madrox »

Tabular-Iceberg wrote: post #27

My concern with this is that in low resource languages the unavoidable biases of the ML models might overpower their own organic development. We shrug off all the little quirks of machine translated text because it usually gets the point across, and we recognize them as quirks because most of what we read was written by real people with no such quirks. But when most of what you read contain those quirks, I fear those…

This happens without machine translation in the wild already with pidgin. If you want to see real life pidgin in action, watch korean and english gamers interact in FPS games. This has been common at the borders of cultures where two languages interact.

Point being, I'm not sure if language purity is more valuable than functionally allowing its people to interact with things they couldn't otherwise. Put another way, should we leave these people locked out of many online resources they can't read because we fear of corrupting their language? Give these people the option and let them decide. Language evolves over time anyway.

Re: No Language Left Behind

#74

Post by david_allison »

jw4ng wrote: post #19

Jeff Wang here with my fellow Meta AI colleague Angela Fan from No Languages left Behind, seeing the comments flowing through. If you want to ask us anything, go for it!

Hi Jeff,

I currently host the largest collection of bilingual Manx[0] English texts (~1MM words). How would I formally get in contact to chat about the steps to make machine translation available (and would there be grant opportunities available for further production of machine-readable data?)

[0] https://en.wikipedia.org/wiki/Manx_language

Re: No Language Left Behind

#75

Post by zen_1 »

nemothekid wrote: post #70
zen_1 wrote: post #60

Earlier quoted context omitted.

Doesn't seem to be a big issue for Arabic, where verbs are gendered (so in the sentence "I am going to the store", the verb "to go" will be either masculine or feminine, reflecting the speaker's gender).

Arabic is the 5th or 6th most spoken language. I think the concern for low resource languages is that nuances like that won't get picked up.

That's fair, I was mostly just responding to the parent comment's point about language models running into potential difficulties in languages where the men and women speak differently (though I don't speak Dakota, so the gender-specific differences there may be more pronounced than in Arabic, where there's also the "default"/neutral option of just picking the masculine version of verbs unless you know the subject(s) are female).

Re: No Language Left Behind

#76

Post by zzzeek »

So glad it's Facebook doing this and not some other weird company, when translating and delivering information to every culture on the planet it's good to have a trustworthy, ethical company without any past (or heck, even any current, ongoing) issues in spreading misinformation around the globe and contributing to the rise of fascism across the world while profiting massively off of it and denying any culpability, making sure it all goes smoothly.

Re: No Language Left Behind

#77

Post by gfaster »

mikewarot wrote: post #16

The analogy I like the most is that they've found the "shape" of languages in high dimensions, and if you rotate the shape for English the right way, you get an unreasonably good fit for the shape of Spanish, again for all the other languages. We're at a point where it's now possible to determine the shape of every language, provided there are enough speakers of the language left who are both able and willing to help…

Anyone who knows or is learning another language can easily tell you that the "warping" methodology of MTL is insufficient. There was a really good video by Tom Scott [1] that talked about this but the short version is that there is critical bits of language in context and inferred by speakers. Any accurate MTL needs nearly full context both on the page and in the cultural moment, in addition to probably needing to ask questions of the author.

[1]: https://www.youtube.com/watch?v=GAgp7nXdkLU

Re: No Language Left Behind

#78

Post by samatman »

labrador wrote: post #66

I'll know AI translators are any good when the United Nations starts using them "Skills required: United Nations translators are required to have a perfect command of their main language and an excellent knowledge of, in most cases, two other official languages" https://www.un.org/dgacm/en/content/translation

An organization built out of pure prestige, with no concept of monetary profit, has zero pressure to stop employing their classmates as translators, ever.

Re: No Language Left Behind

#79

Post by pdonis »

tl/dr: Now your words can be misconstrued by far more people than before, because AIs will translate the misunderstandings into as many languages as possible.

Re: No Language Left Behind

#80

Post by hello_im_angela »

david_allison wrote: post #74
jw4ng wrote: post #19

Jeff Wang here with my fellow Meta AI colleague Angela Fan from No Languages left Behind, seeing the comments flowing through. If you want to ask us anything, go for it!

Hi Jeff, I currently host the largest collection of bilingual Manx[0] English texts (~1MM words). How would I formally get in contact to chat about the steps to make machine translation available (and would there be grant opportunities available for further production of machine-readable data?) [0] https://en.wikipedia.org/wiki/Manx_language

could you send me an email please? It's available on our paper, page 1: https://research.facebook.com/publications/no-language-left-...

Regarding grants: we have offered compute grants previously with the Workshop on Machine Translation (last year: https://www.statmt.org/wmt21/flores-compute-grants.html, this year: https://statmt.org/wmt22/large-scale-multilingual-translatio...) and we have an RFP, but it's currently focused on African languages: https://ai.facebook.com/research/request-for-proposals/trans...

Post reply on HN