Live data from Hacker News

Using AI to match human performance in translating news from Chinese to English

blogs.microsoft.com

21–30 of 62 posts

Re: Using AI to match human performance in translating news from Chinese to English

#22

Compare this to one of Google's blog post promoting their MT research: https://research.googleblog.com/2016/09/a-neural-network-for... It is: 1) More accurate, compared to hyperbole like e.g. "Bridging the Gap between Human and Machine Translation" we have right there in the title the domain: news. 2) A more impressive result. This result is on an independently set up evaluation framework, compared to Google's which…

Googles was nearly 2 years ago though... Thats a long time in this field

Re: Using AI to match human performance in translating news from Chinese to English

#23
About translators solely reliant on NN. The thing is, while 70% of output can be well passable, some of the rest can be very weird if original input was not learned. Like a string of gibberish turning into 10 full sentences.

You have to score the extent of wrongness too.

Re: Using AI to match human performance in translating news from Chinese to English

#24
post #17
post #4

Earlier quoted context omitted.

I think Google translate already has this. I was translating some text into German a few days ago, and after a few sentences I used a word that made it clear that I was talking about a specific type of contract appointment, and it went back and adjusted earlier sentences to use more precise terminology. You only notice this when you a) speak the language you're translating into somewhat; b) actually type/compose the…

That sounds extremely interesting. I had not noticed that feature before. Do you happen to have some example input at hand that triggers such an adjustment?

It's likely this tech is released to only a small percentage of users, and at off-peak times.

Parsing an entire paragraph for context is expensive...

Re: Using AI to match human performance in translating news from Chinese to English

#25

I find these types of "match human performance" claims to be ridiculous, especially when it comes to Chinese -> English translations. Translation is both an art and a science, requiring nuanced understanding of the languages, cultures, and context. It also demands quite a bit of creativity. No translation tool I've tried has come even close to matching human performance of a good human translator, including microsoft…

I think there are a lot of bad human translators out there...

The number of papers I've read that have very poor grammar, to such an extent it's barely understandable...

Re: Using AI to match human performance in translating news from Chinese to English

#26
post #6

The most impressive ML translation tool I've seen so far is DeepL[0]. Sometimes, it manages to translate whole articles without errors. [0]: https://www.deepl.com/translator

I can also confirm, it's very impressive. Not sure how their setup differs from Google Translate but I've read articles translated with it and only knew it was translated when it was written at the end of the article that it was using DeepL.

Word choice is sometimes very slightly off, but grammar appears to be flawless. As you say, very very impressive.

Re: Using AI to match human performance in translating news from Chinese to English

#28
post #14
post #8

As impressive as it may be, these people should refrain from claiming 'human-like' translation from a system that has no way of 'knowing' anything about context, other than statistical occurrences. It is certain that, on occasion, the system will make such mistakes as stating the opposite of what is being said in the first place, or attribute one action to the wrong person, and what not. Perhaps on average it's as go…

This was much the point that John Searle made in his criticism of AI prognostications. Ultimately I think he'll be shown to be wrong but the time frame for this will be (I suggest) much longer than is currently touted. https://en.wikipedia.org/wiki/John_Searle

Are you referring to the Chinese Room? I've always had an issue with that argument. Instructions are immutable, but neural networks certainly are not.

Re: Using AI to match human performance in translating news from Chinese to English

#29
post #6

The most impressive ML translation tool I've seen so far is DeepL[0]. Sometimes, it manages to translate whole articles without errors. [0]: https://www.deepl.com/translator

Impressive, but there's an easy formula for getting these systems to make mistakes. Just input a sentence with some kind of long distance dependency. For example, DeepL gets agreement right in English to Spanish translations when the two things that agree are close together:

    I like soup -> COMO sopa
    They eat soup -> COMEN sopa
Impressively, it can even get agreement correct across clause boundaries in many cases. But if you do wh-movement through two or more clauses, you're usually out of luck:

    Which boys does he say he believes eat soup?
    ->
    ¿Qué chicos dice que cree que COME sopa? [should be COMEN]
It doesn't really matter very much in practice if an MT system makes mistakes like this, but they are mistakes that you can rely on humans not to make systematically.

Re: Using AI to match human performance in translating news from Chinese to English

#30
post #8

As impressive as it may be, these people should refrain from claiming 'human-like' translation from a system that has no way of 'knowing' anything about context, other than statistical occurrences. It is certain that, on occasion, the system will make such mistakes as stating the opposite of what is being said in the first place, or attribute one action to the wrong person, and what not. Perhaps on average it's as go…

Let's just define that "human like" in context of machine translation from now on mean "with full legal responsibility". Then let's see who claim their translator is "human like".
Post reply on HN