Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

No Language Left Behind

ai.facebook.com

Post reply on HN
Print view View original

151–160 of 166 posts

Re: No Language Left Behind

#151

Post by bertil »

otreblatercero wrote: post #150
bertil wrote: post #138

Earlier quoted context omitted.

Are there online corpora, like Wikipedia, that could be used to train the models? Are those under a permissive enough license to be used for model training? If there are spoken, with enough budget, a library of voices could be recorded. I think you’d prefer that collection to be gathered and maintained by a non-profit rather than Meta.

For náhuatl, I found this: Wikipedia in nahuatl https://nah.wikipedia.org/wiki/Cal%C4%ABxatl

I’m wondering if 7065 articles is enough to train the model.

Re: No Language Left Behind

#152

Post by jw4ng »

kouteiheika wrote: post #111
jw4ng wrote: post #19

Jeff Wang here with my fellow Meta AI colleague Angela Fan from No Languages left Behind, seeing the comments flowing through. If you want to ask us anything, go for it!

If your goal is to make inclusive translation more widely available why license the models under a non-commercial license? This basically makes it impossible to use legally (or at least without a lot of legal risk) for essentially anyone due to the vague definition of what's commercial. Is Facebook hurting for money and looking to commercially license this model on request?

This enables any researcher to use our code freely, and build on top of it for their own research. We are not intending to commercially licensing our project.

Re: No Language Left Behind

#153

Post by jw4ng »

yosito wrote: post #117
jw4ng wrote: post #19

Jeff Wang here with my fellow Meta AI colleague Angela Fan from No Languages left Behind, seeing the comments flowing through. If you want to ask us anything, go for it!

I'm curious how much work it takes to prepare training data for a language. From anecdotal experience, I've always been able to learn some basic survival skills in a new language by studying the translations of about 20 key phrases for a week or so, which give me the ability to combine them into a few hundred different phrases and survive most daily transactions. So I always imagine that training a language model is…

Without any supervised training data, it's pretty difficult to create a very good translation model. For many languages, data might only be available through religious domains such as the Bible, or not available at all. We created a dataset called NLLB-Seed for this reason --- it's approx 6K sentences available for 39 languages that translates a broad set of topics from Wikipedia. We found that with a dataset like NLLB-Seed, we're able to have sufficient supervised signal to jumpstart our automatic dataset creation pipeline. Of course, the more high quality aligned data the better the model performance, but our project explores how we can make models more efficient at learning even when the training data is small.

Importantly, models can learn from other languages are similar. If we train separate models for each direction on small amounts of data, the performance is significantly worse than grouping languages in one large multilingual model.

Re: No Language Left Behind

#154

Post by [deleted] »

mensetmanusman wrote: post #141
protomyth wrote: post #91

Earlier quoted context omitted.

I'll take that in good faith, but I will say Facebook has been a particular pain for many tribal folks given its true name policy and banning people who it thinks are using a fake name. Yellow Horse was one that was wildly reported, but their are others. Mostly anything that takes the form Adjective Noun. Had a rather painful thread with someone claiming to be a Facebook employee that defended this practice. I haven'…

I was unfamiliar with this issue. Was their name Yellow Horse in English? Or was it supposed to be written in a language not available so an English translation was used? I have a feeling that if it was written in the original language it would go through, since many English names also have adjective noun original meanings like ‘beautiful flower’.

[deleted]

Re: No Language Left Behind

#155

Post by [deleted] »

zen_1 wrote: post #75
nemothekid wrote: post #70

Earlier quoted context omitted.

Arabic is the 5th or 6th most spoken language. I think the concern for low resource languages is that nuances like that won't get picked up.

That's fair, I was mostly just responding to the parent comment's point about language models running into potential difficulties in languages where the men and women speak differently (though I don't speak Dakota, so the gender-specific differences there may be more pronounced than in Arabic, where there's also the "default"/neutral option of just picking the masculine version of verbs unless you know the subject(s)…

[deleted]

Re: No Language Left Behind

#156

Post by concinds »

jw4ng wrote: post #19

Jeff Wang here with my fellow Meta AI colleague Angela Fan from No Languages left Behind, seeing the comments flowing through. If you want to ask us anything, go for it!

These initiatives are always couched in "inclusion" rhetoric (the very name of your project is telling); I don't doubt for a second that it's a genuine sentiment, but I strongly suspect your team hasn't thought through the full, self-defeating implications of universal language translation.

The problem is that it increases the risk of monoculture to 100%. Without language barriers, cultural diversity is lost, not gained, since you have winner-take-all effects[0]. Instead of helping revive languages, it'll make American ideas, mores, morality (Puritanism), philosophies, and political values more dominant worldwide.

To be clear, this will increase economic opportunity, but will inevitably kill cultural diversity.

Is your team considering or studying this?

[0]: https://www.sampaxinos.com/mental-models/the-winner-takes-al... (or see Taleb's works)

Re: No Language Left Behind

#157

Post by protomyth »

mensetmanusman wrote: post #141
protomyth wrote: post #91

Earlier quoted context omitted.

I'll take that in good faith, but I will say Facebook has been a particular pain for many tribal folks given its true name policy and banning people who it thinks are using a fake name. Yellow Horse was one that was wildly reported, but their are others. Mostly anything that takes the form Adjective Noun. Had a rather painful thread with someone claiming to be a Facebook employee that defended this practice. I haven'…

I was unfamiliar with this issue. Was their name Yellow Horse in English? Or was it supposed to be written in a language not available so an English translation was used? I have a feeling that if it was written in the original language it would go through, since many English names also have adjective noun original meanings like ‘beautiful flower’.

Was their name Yellow Horse in English? Yes

I have a feeling that if it was written in the original language it would go through, since many English names also have adjective noun original meanings like ‘beautiful flower’. The legal name is in English, so anyone expecting it to be written in the original language is expecting too much.

Re: No Language Left Behind

#158

Post by kouteiheika »

jw4ng wrote: post #152
kouteiheika wrote: post #111

Earlier quoted context omitted.

If your goal is to make inclusive translation more widely available why license the models under a non-commercial license? This basically makes it impossible to use legally (or at least without a lot of legal risk) for essentially anyone due to the vague definition of what's commercial. Is Facebook hurting for money and looking to commercially license this model on request?

This enables any researcher to use our code freely, and build on top of it for their own research. We are not intending to commercially licensing our project.

Okay, but why?

If your aim is to make this technology more widely available and, as you claim, "give people the opportunity to access and share web content in their native language, and communicate with anyone, anywhere, regardless of their language preferences", then why make it so that the model essentially can't be used for anything useful? It doesn't really make any sense.

Legally even the use case which you're promoting on your frontpage - the Wikipedia Foundation's Content Translation - is illegal under the non-commercial license in certain jurisdictions! For example, see here: https://www.techdirt.com/2014/03/27/german-court-says-creati...

Even using it for research would be illegal as it's also not exactly "personal use".

Re: No Language Left Behind

#159

Post by cyphar »

raverbashing wrote: post #133
cyphar wrote: post #125

Earlier quoted context omitted.

There are several reasons a language can go extinct, and while the reasons you mentioned are included there are several other reasons that you didn't consider. The most common way language death occurs is through contact with a prestige language, which results in social pressure that makes the non-prestige language become less and less commonly spoken[1]. A community becoming bilingual is part of this process -- it d…

Icelandic won't disappear soon. (it's a bit of a mistake to say latin disappeared but I digress) It might take some work (and more media being produced in it, surely) but I don't think it's at risk > because smartphones do not support their native language Nowhere in the article says that. And most smartphones do even have support (including the Icelandic keyboard) what is not supported is voice activated devices: >…

(I guess we've skipped over the discussion about whether language death can occur without genocide.)

> it's a bit of a mistake to say latin disappeared but I digress

Latin is a dead language. It's not extinct (because it's used for ceremonial reasons by the Catholic Church) but it has no native speakers. This is not a remotely controversial statement. Old Church Slavonic has a similar role in Slavic Orthodox Churches and is also a dead language for the same reason.

> Nowhere in the article says that. And most smartphones do even have support (including the Icelandic keyboard).

In my defense, I can't read the article -- I linked it because this Tom Scott video[1] uses that article as a reference for the Icelandic example he gave where he explicitly says that Icelandic is not supported by modern smartphones. My phone does have Icelandic support now so I guess the statement was only true at the time he said it?

[1]: https://youtu.be/qYlmFfsyLMo?t=137

Re: No Language Left Behind

#160

Post by cyphar »

MrPatan wrote: post #137
eCa wrote: post #124

Earlier quoted context omitted.

> prunes out as many languages as possible Would you feel the same way if that includes all languages you know, including all versions of English?

How would you feel about being the one that has to teach your children a language that will hinder their prospects instead of one that will help them succeed, just so that the speakers of the bigger language feel good about themselves that they are good people or something?

I could not think of a more obvious way of telling everyone that you are monolingual than implying that knowing another language is a burden. Children are not burdened by being multilingual.

It is true that social pressures kill off local languages, but it's usually not because parents don't want to teach their children their mother tongue, it's that people stop using the language to communicate because of the influence of the "prestige language". My parents (and all of the parents in the immigrant community I live in) went through great pains to teach their children their native language.

Post reply on HN