Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using AI to match human performance in translating news from Chinese to English

blogs.microsoft.com

Post reply on HN
Print view View original

41–50 of 62 posts

Re: Using AI to match human performance in translating news from Chinese to English

#41

Post by Analemma_ »

dvh wrote: post #30
d--b wrote: post #8

As impressive as it may be, these people should refrain from claiming 'human-like' translation from a system that has no way of 'knowing' anything about context, other than statistical occurrences. It is certain that, on occasion, the system will make such mistakes as stating the opposite of what is being said in the first place, or attribute one action to the wrong person, and what not. Perhaps on average it's as go…

Let's just define that "human like" in context of machine translation from now on mean "with full legal responsibility". Then let's see who claim their translator is "human like".

If you defined it that way, not even human translators would meet the standard. Treaties and other official documents published in multiple languages always specify one as the "official" one for purposes of legal interpretation and that, in the event of conflict or confusion, the translations are subservient to it. Setting a bar for AI performance so high that even humans don't reach it seems unhelpful.

Re: Using AI to match human performance in translating news from Chinese to English

#42

Post by occamrazor »

Analemma_ wrote: post #41
dvh wrote: post #30

Earlier quoted context omitted.

Let's just define that "human like" in context of machine translation from now on mean "with full legal responsibility". Then let's see who claim their translator is "human like".

If you defined it that way, not even human translators would meet the standard. Treaties and other official documents published in multiple languages always specify one as the "official" one for purposes of legal interpretation and that, in the event of conflict or confusion, the translations are subservient to it. Setting a bar for AI performance so high that even humans don't reach it seems unhelpful.

Actually most international treaties specify all language versions to be equally authentic. Multilingual contracts on the other hand generally have a single authoritative version.

Re: Using AI to match human performance in translating news from Chinese to English

#43

Post by blennon »

What I find most interesting is the multiple training methods used to get the network to improve its performance. They name a few in the article:

- dual learning - deliberation networks - joint training - agreement regularization

I haven't read the paper to see how these are combined but it makes intuitive sense that using multiple training methods can lead to better performance. That is to say, to more effectively search the weight space of the network.

Re: Using AI to match human performance in translating news from Chinese to English

#44

Post by romaniv »

Chathamization wrote: post #32
avgarrison wrote: post #28

Earlier quoted context omitted.

Are you referring to the Chinese Room? I've always had an issue with that argument. Instructions are immutable, but neural networks certainly are not.

I agree, the argument seems to come down to saying that human consciousness can't be replicated because human consciousness stems from a "soul" (a non-physical and undetectable element of someone that's at the root of their consciousness). The amount of attention this argument has received has made me wonder whether the "rigor" used by philosophy departments is mostly just a way to obfuscate bad arguments.

That's not his argument at all. His argument is that just because you can do some task doesn't mean you "understand" it.

You don't need some fancy philosophy and complex thought experiments to see what he means. Just look at how people learn math. You can do calculations by memorizing algebraic rules, but that's not the same as understanding why those rules exist and what they mean. Even though you will calculate answers correctly in both cases, we all know there is a qualitative difference between them.

Back to Searle. His argument is that everything computers do is analogous to rote memorization and that transition to understanding requires something computers don't have.

Whether you buy his argument, two things are clear. First, there is a difference between just producing results and understanding the process. We all experienced this difference. It's all theoretical as long as you stick to simple tests (like multiple-choice exams), but becomes relevant when you suddenly expand the context (like requiring the student to prove some theorem instead of doing a calculation). Second, we also know that for humans this difference isn't just quantitative. Memorizing more algebraic rules and training in their application will not automatically result in students gaining understanding of mathematical principles.

Re: Using AI to match human performance in translating news from Chinese to English

#45

Post by romaniv »

anewhnaccount2 wrote: post #3

Compare this to one of Google's blog post promoting their MT research: https://research.googleblog.com/2016/09/a-neural-network-for... It is: 1) More accurate, compared to hyperbole like e.g. "Bridging the Gap between Human and Machine Translation" we have right there in the title the domain: news. 2) A more impressive result. This result is on an independently set up evaluation framework, compared to Google's which…

Microsoft Research also seems to be much more willing to create usable demos of their work.

Re: Using AI to match human performance in translating news from Chinese to English

#46

Post by foldr »

foldr wrote: post #29
lima wrote: post #6

The most impressive ML translation tool I've seen so far is DeepL[0]. Sometimes, it manages to translate whole articles without errors. [0]: https://www.deepl.com/translator

Impressive, but there's an easy formula for getting these systems to make mistakes. Just input a sentence with some kind of long distance dependency. For example, DeepL gets agreement right in English to Spanish translations when the two things that agree are close together: I like soup -> COMO sopa They eat soup -> COMEN sopa Impressively, it can even get agreement correct across clause boundaries in many cases. But…

(outside edit window. first sentence should be 'I eat soup' not 'I like soup')

Re: Using AI to match human performance in translating news from Chinese to English

#47

Post by tinymollusk »

d--b wrote: post #8

As impressive as it may be, these people should refrain from claiming 'human-like' translation from a system that has no way of 'knowing' anything about context, other than statistical occurrences. It is certain that, on occasion, the system will make such mistakes as stating the opposite of what is being said in the first place, or attribute one action to the wrong person, and what not. Perhaps on average it's as go…

Humans, on the other hand, have all kinds of biases (intentional and unintentional, conscious or not) that creep in because they do have context.

The important thing is that we build systems that account for the process-based problems, not that we build components that are perfect. Things that matter more are how frequently these mistakes happen? What's the impact? Can we eliminate these errors with multiple layers of processes designed to identify the exploits?

Re: Using AI to match human performance in translating news from Chinese to English

#48

Post by echevil »

foldr wrote: post #29
lima wrote: post #6

The most impressive ML translation tool I've seen so far is DeepL[0]. Sometimes, it manages to translate whole articles without errors. [0]: https://www.deepl.com/translator

Impressive, but there's an easy formula for getting these systems to make mistakes. Just input a sentence with some kind of long distance dependency. For example, DeepL gets agreement right in English to Spanish translations when the two things that agree are close together: I like soup -> COMO sopa They eat soup -> COMEN sopa Impressively, it can even get agreement correct across clause boundaries in many cases. But…

AI researchers are certainly fully aware of such issues, and there have been various models trying to encode long term dependency information, e.g. LSTM.

Re: Using AI to match human performance in translating news from Chinese to English

#49

Post by dmfdmf »

romaniv wrote: post #44
Chathamization wrote: post #32

Earlier quoted context omitted.

I agree, the argument seems to come down to saying that human consciousness can't be replicated because human consciousness stems from a "soul" (a non-physical and undetectable element of someone that's at the root of their consciousness). The amount of attention this argument has received has made me wonder whether the "rigor" used by philosophy departments is mostly just a way to obfuscate bad arguments.

That's not his argument at all. His argument is that just because you can do some task doesn't mean you "understand" it. You don't need some fancy philosophy and complex thought experiments to see what he means. Just look at how people learn math. You can do calculations by memorizing algebraic rules, but that's not the same as understanding why those rules exist and what they mean . Even though you will calculate an…

Thanks for rebutting Chathamization's gross misrepresentation of Searle's argument.

> It's all theoretical as long as you stick to simple tests (like multiple-choice exams), but becomes relevant when you suddenly expand the context (like requiring the student to prove some theorem instead of doing a calculation).

Not even expanding the context changes the situation. The proof of a theorem can be memorized without any understanding just as easily as algebraic rules.

> Second, we also know that for humans this difference isn't just quantitative. Memorizing more algebraic rules and training in their application will not automatically result in students gaining understanding of mathematical principles.

This is correct and the same principle applies not just to humans but to computers too (which was the point of Searle's argument). No amount of computation is going to make a computer aware or understand the meaning of the symbols. Ultimately "meaning" is our perceptual awareness of existence but that is a long proof for another day.

Re: Using AI to match human performance in translating news from Chinese to English

#50

Post by red75prime »

dvh wrote: post #30
d--b wrote: post #8

As impressive as it may be, these people should refrain from claiming 'human-like' translation from a system that has no way of 'knowing' anything about context, other than statistical occurrences. It is certain that, on occasion, the system will make such mistakes as stating the opposite of what is being said in the first place, or attribute one action to the wrong person, and what not. Perhaps on average it's as go…

Let's just define that "human like" in context of machine translation from now on mean "with full legal responsibility". Then let's see who claim their translator is "human like".

Well, Hiroshima and Nagasaki bombing was allegedly due to a translation error. It will certainly take time for a machine translation to catch up.
Post reply on HN