The transformer paper was quite influential in machine translation space. This resource [0] posted here a while back is a good place to learn and get a better idea how it works.
Building a language translator from scratch with deep learning
11–20 of 38 posts
Re: Building a language translator from scratch with deep learning
#12The transformer paper was quite influential in machine translation space. This resource [0] posted here a while back is a good place to learn and get a better idea how it works. [0]: http://nlp.seas.harvard.edu/2018/04/03/attention.html
one of the best visual tutorials on the transformer I came across http://jalammar.github.io/illustrated-transformer/
Re: Building a language translator from scratch with deep learning
#13Machine translation has made some pretty impressive progress over the last decade. Unfortunately no methods will ever cover the very last mile as languages don't have perfect 1 to 1 mappings. Though it is amusing watching the machines try.
Re: Building a language translator from scratch with deep learning
#14Machine translation has made some pretty impressive progress over the last decade. Unfortunately no methods will ever cover the very last mile as languages don't have perfect 1 to 1 mappings. Though it is amusing watching the machines try.
If we ever do solve the last mile, it would probably be one of the less interesting consequences, as it would probably imply we've built an algorithm capable of learning and thinking to a similar degree of a human.
To that, though, I'm definitely not holding my breath :)
Re: Building a language translator from scratch with deep learning
#15Machine translation has made some pretty impressive progress over the last decade. Unfortunately no methods will ever cover the very last mile as languages don't have perfect 1 to 1 mappings. Though it is amusing watching the machines try.
If we ever do solve the last mile, it would probably be one of the less interesting consequences, as it would probably imply we've built an algorithm capable of learning and thinking to a similar degree of a human. To that, though, I'm definitely not holding my breath :)
>>To that, though, I'm definitely not holding my breath :)
Nature was able to do this. Sure, it took a couple billion years of evolution to get to this point, but it is doable. I'm betting that the chances of us inventing Strong AI within the next 100 years is almost a near certainty.
Re: Building a language translator from scratch with deep learning
#16The grammar correction in Google Translate is a little too good. I was trying to create some broken Russian phrases to send a Russian friend, but I’d put in weird or bad English as an input and get very good Russian as an output!
Re: Building a language translator from scratch with deep learning
#17The transformer paper was quite influential in machine translation space. This resource [0] posted here a while back is a good place to learn and get a better idea how it works. [0]: http://nlp.seas.harvard.edu/2018/04/03/attention.html
one of the best visual tutorials on the transformer I came across http://jalammar.github.io/illustrated-transformer/
Wow, that does look really good! Thanks!
Re: Building a language translator from scratch with deep learning
#18Re: Building a language translator from scratch with deep learning
#19This is very cool. One thing I wonder about though is whether small companies will be able to compete with large ones like Google in ML in the future. One reason Google's translator is better is because they have way more data. In the past they digitized tons of books so they have an excellent dataset that has been translated by professional, human translators. This data collection is effectively cross-subsidized by…
EU helps with this too, accidentally. All official documents are translated in all EU languages, with very high quality translators. And all these documents are public.
Re: Building a language translator from scratch with deep learning
#20This is very cool. One thing I wonder about though is whether small companies will be able to compete with large ones like Google in ML in the future. One reason Google's translator is better is because they have way more data. In the past they digitized tons of books so they have an excellent dataset that has been translated by professional, human translators. This data collection is effectively cross-subsidized by…
The value of large corpora for translation may be diminishing.. In particular, Facebook have achieved impressive results using unsupervised ML for translation: https://code.fb.com/ai-research/unsupervised-machine-transla... The basic idea is to use word vector embeddings to build a source target dictionary, then combine this with a language recognition model to iteratively bootstrap a set of source target training ex…
So the value of a large corpus remains, it's just this one happens to be generated, as opposed to collected.