Firefox is getting language translation • Hacker News Board

Re: Firefox is getting language translation

#71

Post by yorwba » Tue, Oct 22, 2019, 12:49 PM UTC

mantap wrote: post #60

amirmasoudabdol wrote: post #4

While this is great, I like to give a shout out to DeepL Translator [1]. I'm not affiliated with them but I like to recommend them to people who like to step out of Google ecosystem. I am using DeepL for about a year now and I mostly use it for NL EN, DE EN. So far, I never felt that the translation is off, or terrible, and if not better in some cases, it's as good as Google Translator. [1]: https://www.deepl.com/tra…

It looks useful but the lack of non-European languages makes me slightly suspicious, I wonder if their approach generalises to Arabic, Chinese, Japanese, etc.

It depends on which part of their approch you focus on. I'd expect their machine translation model to be sufficiently general to support basically any language given enough training data. Yes, the languages you listed have some edge cases, but so do languages they already support fine. For example, the lack of spaces separating words in Chinese and Japanese can be handled by the same word-piece segmentation they need for German compound words.

The bigger problem is likely to be lack of training data. Unless they have a pile of cash to pay professional translators to produce a parallel corpus, the alternative is to scrape translations from the internet. Basically, crawl the same site multiple times with different Accept-Language headers and try to align the results. Crucially, this depends on an existing ecosystem of bilingual websites with high-quality human translations.

According to DeepL's website they're a spin-off of Linguee, who provide a search service for exactly that kind of parallel data. So before DeepL starts supporting any given language pair, you should expect it to appear in Linguee first. https://en.wikipedia.org/wiki/Linguee

Edit: It took me a while to figure out how to select a different language on https://linguee.com (Their UI seems broken using mobile Firefox Preview.) Appending /english-chinese and /english-japanese to the URL shows that they already support those two, and the alignment of translations appears reasonable to me. No /english-arabic, though.

Re: Firefox is getting language translation

#72

Post by ktpsns » Tue, Oct 22, 2019, 12:53 PM UTC

It should be noted that this project is funded by the European union. You can find all funding details, reports and publications at https://cordis.europa.eu/project/rcn/219608/factsheet/en (the project started in January 2019, so there are not much reports yet).

It is amazing that they work so closely together with Firefox, so the project result will be really of a use for not only European citizens but people on all the world.

Re: Firefox is getting language translation

#73

Post by darkwater » Tue, Oct 22, 2019, 1:08 PM UTC

NilsIRL wrote: post #42

darkwater wrote: post #11

I think that all the "hidden in the cloud" services like what Google Photos does, translations, well basically every service Google has, must go "edge computing" in the next years. The hardware side is already powerful enough, even in cheap Android smartphones, we just miss a better, dedicated software. Basically an Android phone without many Google Services, although there are some, like Google identity & authentica…

Except that would allow people to reverse engineer the code which I think is one the main reasons there is still a lot of services that are in the cloud.

AFAIK Apple already does lot of local device post processing and nobody broke it

Re: Firefox is getting language translation

#74

Post by als0 » Tue, Oct 22, 2019, 1:19 PM UTC

This is great. I hate relying on the translation services of Microsoft and Google, but I’ve been using them since they’re typically more convenient to use than separate applications.

Re: Firefox is getting language translation

#75

Post by _emacsomancer_ » Tue, Oct 22, 2019, 1:28 PM UTC

distances wrote: post #28

amirmasoudabdol wrote: post #4

While this is great, I like to give a shout out to DeepL Translator [1]. I'm not affiliated with them but I like to recommend them to people who like to step out of Google ecosystem. I am using DeepL for about a year now and I mostly use it for NL EN, DE EN. So far, I never felt that the translation is off, or terrible, and if not better in some cases, it's as good as Google Translator. [1]: https://www.deepl.com/tra…

> So far, I never felt that the translation is off, or terrible, and if not better in some cases, it's as good as Google Translator. In my experience Deepl is consistently, without fail, considerably better than Google Translate. I basically use Google now only for more exotic language pairs, or full-page translations.

> more exotic language pairs

I've found that sometimes what Google does for these is translate from Language A > English, and then English > Language B, which leads to bizarre results.

Re: Firefox is getting language translation

#76

Post by dredmorbius » Tue, Oct 22, 2019, 1:36 PM UTC

amirmasoudabdol wrote: post #4

While this is great, I like to give a shout out to DeepL Translator [1]. I'm not affiliated with them but I like to recommend them to people who like to step out of Google ecosystem. I am using DeepL for about a year now and I mostly use it for NL EN, DE EN. So far, I never felt that the translation is off, or terrible, and if not better in some cases, it's as good as Google Translator. [1]: https://www.deepl.com/tra…

I'd meant to add this to my DDG !bang list yesterday: !deepl

Re: Firefox is getting language translation

#77

Post by [deleted] » Tue, Oct 22, 2019, 1:36 PM UTC

_emacsomancer_ wrote: post #75

distances wrote: post #28

Earlier quoted context omitted.

> So far, I never felt that the translation is off, or terrible, and if not better in some cases, it's as good as Google Translator. In my experience Deepl is consistently, without fail, considerably better than Google Translate. I basically use Google now only for more exotic language pairs, or full-page translations.

> more exotic language pairs I've found that sometimes what Google does for these is translate from Language A > English, and then English > Language B, which leads to bizarre results.

[deleted]

Re: Firefox is getting language translation

#78

Post by dredmorbius » Tue, Oct 22, 2019, 1:39 PM UTC

_emacsomancer_ wrote: post #75

distances wrote: post #28

Earlier quoted context omitted.

> So far, I never felt that the translation is off, or terrible, and if not better in some cases, it's as good as Google Translator. In my experience Deepl is consistently, without fail, considerably better than Google Translate. I basically use Google now only for more exotic language pairs, or full-page translations.

> more exotic language pairs I've found that sometimes what Google does for these is translate from Language A > English, and then English > Language B, which leads to bizarre results.

From reports, Google Translate has effectively created its own internal (AI GD ML) metalanguage, which can interpolate between languages it's not been specifically trained on. E.g., with Japanese English and Korean English, Google Translate can manage Japanese Korean, without being specifically trained to do so.

So yes, there's an intermediary language. But it's not English.

https://www.newscientist.com/article/2114748-google-translat...

Re: Firefox is getting language translation

#79

Post by azepoi » Tue, Oct 22, 2019, 1:42 PM UTC

yorwba wrote: post #71

mantap wrote: post #60

Earlier quoted context omitted.

It looks useful but the lack of non-European languages makes me slightly suspicious, I wonder if their approach generalises to Arabic, Chinese, Japanese, etc.

It depends on which part of their approch you focus on. I'd expect their machine translation model to be sufficiently general to support basically any language given enough training data. Yes, the languages you listed have some edge cases, but so do languages they already support fine. For example, the lack of spaces separating words in Chinese and Japanese can be handled by the same word-piece segmentation they need…

The European Union, the Swiss confederation, Belgium, Canada and other multilingual states, the European patent office and many international organization provide a huge corpus of professionally translated documents and reports for major European languages. Not so much for Japanese.

Re: Firefox is getting language translation

#80

Post by bzbarsky » Tue, Oct 22, 2019, 2:00 PM UTC

dredmorbius wrote: post #78

_emacsomancer_ wrote: post #75

Earlier quoted context omitted.

> more exotic language pairs I've found that sometimes what Google does for these is translate from Language A > English, and then English > Language B, which leads to bizarre results.

From reports, Google Translate has effectively created its own internal (AI GD ML) metalanguage, which can interpolate between languages it's not been specifically trained on. E.g., with Japanese English and Korean English, Google Translate can manage Japanese Korean, without being specifically trained to do so. So yes, there's an intermediary language. But it's not English. https://www.newscientist.com/article/21147…

The intermediary language is not English, but due to the way the training set is constructed (pairs of texts in various languages, with the vast majority of pairs having English as one of the languages, is my understanding) it can be very hard to tell apart from English sometimes.

For example, translating "рубанок" ("plane", in the sense of the carpenter's tool) from Russian to Polish used to produce "samolot" ("airplane") in Google translate up until sometime earlier this year, because in the intermediate representation "plane" was ambiguous just like it is in English. It looks like that particular bit is fixed now, which is at least progress! Maybe they've been adding more non-English text pairs...