Using AI to match human performance in translating news from Chinese to English
1–10 of 62 posts
Re: Using AI to match human performance in translating news from Chinese to English
#2Translate "sentences of news" is very different to translating an entire article, which is obviously what's interesting.
Is anybody in MT or text comprehension/generation really working on systems that construct a model/"understanding" of the bigger narrative in a longer-running text? Even just to be able to do correct anaphora resolution across sentence and paragraph boundaries, but intuitively also WSD seems easier if you've got some sort of abstract context over more than just a sentence.
Re: Using AI to match human performance in translating news from Chinese to English
#3Compare this to one of Google's blog post promoting their MT research: https://research.googleblog.com/2016/09/a-neural-network-for...
It is:
1) More accurate, compared to hyperbole like e.g. "Bridging the Gap between Human and Machine Translation" we have right there in the title the domain: news.
2) A more impressive result. This result is on an independently set up evaluation framework, compared to Google's which used their own framework.
Compare further the papers: https://arxiv.org/pdf/1609.08144.pdf https://www.microsoft.com/en-us/research/uploads/prod/2018/0...
These researcher appear to have been much clearer about what they're actually claiming, and also used more standard evaluation tools (Appraise) and methodology rather than something haphazardly hacked together.
Re: Using AI to match human performance in translating news from Chinese to English
#4Translate "sentences of news" is very different to translating an entire article, which is obviously what's interesting. Is anybody in MT or text comprehension/generation really working on systems that construct a model/"understanding" of the bigger narrative in a longer-running text? Even just to be able to do correct anaphora resolution across sentence and paragraph boundaries, but intuitively also WSD seems easier…
I think Google translate already has this. I was translating some text into German a few days ago, and after a few sentences I used a word that made it clear that I was talking about a specific type of contract appointment, and it went back and adjusted earlier sentences to use more precise terminology. You only notice this when you a) speak the language you're translating into somewhat; b) actually type/compose the message in the Google Translate text box; and c) are typing something idiomatic enough that such specific phrases can be inferred. So I guess it's just something you wouldn't normally notice.
Either way, I was mightily impressed, to the point where my wife had to roll her eyes and say 'yeah yeah I understand it now' to get me to drop it. (I'm just easily excited I guess.)
Re: Using AI to match human performance in translating news from Chinese to English
#5Translate "sentences of news" is very different to translating an entire article, which is obviously what's interesting. Is anybody in MT or text comprehension/generation really working on systems that construct a model/"understanding" of the bigger narrative in a longer-running text? Even just to be able to do correct anaphora resolution across sentence and paragraph boundaries, but intuitively also WSD seems easier…
People have certainly worked on moving beyond sentence boundaries, although what is meant by understanding is always a bit nebulous. Certainly we need to make sure whatever process we are using has a sufficiently rich internal knowledge representation. One piece of work is this: https://github.com/chardmeier/docent/wiki which is a document level phrases-based statistical machine translation decoder. There have also been special purpose evaluation tasks which include correct pronoun resolution e.g. https://lindat.mff.cuni.cz/repository/xmlui/handle/11372/LRT... .
If you're interested, probably the DiscoMT workshops are a good starting point for some things people have tried.
Re: Using AI to match human performance in translating news from Chinese to English
#6The most impressive ML translation tool I've seen so far is DeepL[0].
Sometimes, it manages to translate whole articles without errors.
Re: Using AI to match human performance in translating news from Chinese to English
#7Be careful when reading such claims: https://www.theatlantic.com/technology/archive/2018/01/the-s...
Re: Using AI to match human performance in translating news from Chinese to English
#8As impressive as it may be, these people should refrain from claiming 'human-like' translation from a system that has no way of 'knowing' anything about context, other than statistical occurrences.
It is certain that, on occasion, the system will make such mistakes as stating the opposite of what is being said in the first place, or attribute one action to the wrong person, and what not. Perhaps on average it's as good as a person, but this system will make mistakes that disqualifies it from being used without a bucket of salt.
Re: Using AI to match human performance in translating news from Chinese to English
#9Would be nice to have improved MTL performance for Wuxia/Xanxia webnovels.
Re: Using AI to match human performance in translating news from Chinese to English
#10As impressive as it may be, these people should refrain from claiming 'human-like' translation from a system that has no way of 'knowing' anything about context, other than statistical occurrences. It is certain that, on occasion, the system will make such mistakes as stating the opposite of what is being said in the first place, or attribute one action to the wrong person, and what not. Perhaps on average it's as go…
Certainly true. I'd just point out that exactly these kinds of mistakes happen to people too.