Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using AI to match human performance in translating news from Chinese to English

blogs.microsoft.com

Post reply on HN
Print view View original

31–40 of 62 posts

Re: Using AI to match human performance in translating news from Chinese to English

#31

Post by foldr »

avgarrison wrote: post #28
vixen99 wrote: post #14

Earlier quoted context omitted.

This was much the point that John Searle made in his criticism of AI prognostications. Ultimately I think he'll be shown to be wrong but the time frame for this will be (I suggest) much longer than is currently touted. https://en.wikipedia.org/wiki/John_Searle

Are you referring to the Chinese Room? I've always had an issue with that argument. Instructions are immutable, but neural networks certainly are not.

Any finite computational system can be implemented as a lookup table in the manner that Searle suggests. But that's not essential to the argument. You can imagine that the man in the room is following instructions for simulating a (finite approximation of) a Turing machine, or any other computational device that you like.

Re: Using AI to match human performance in translating news from Chinese to English

#32

Post by Chathamization »

avgarrison wrote: post #28
vixen99 wrote: post #14

Earlier quoted context omitted.

This was much the point that John Searle made in his criticism of AI prognostications. Ultimately I think he'll be shown to be wrong but the time frame for this will be (I suggest) much longer than is currently touted. https://en.wikipedia.org/wiki/John_Searle

Are you referring to the Chinese Room? I've always had an issue with that argument. Instructions are immutable, but neural networks certainly are not.

I agree, the argument seems to come down to saying that human consciousness can't be replicated because human consciousness stems from a "soul" (a non-physical and undetectable element of someone that's at the root of their consciousness).

The amount of attention this argument has received has made me wonder whether the "rigor" used by philosophy departments is mostly just a way to obfuscate bad arguments.

Re: Using AI to match human performance in translating news from Chinese to English

#33

Post by golfer »

anewhnaccount2 wrote: post #3

Compare this to one of Google's blog post promoting their MT research: https://research.googleblog.com/2016/09/a-neural-network-for... It is: 1) More accurate, compared to hyperbole like e.g. "Bridging the Gap between Human and Machine Translation" we have right there in the title the domain: news. 2) A more impressive result. This result is on an independently set up evaluation framework, compared to Google's which…

If you are claiming that Microsoft is pure as the driven snow with regard to making exaggerated and hyperbolic claims, then you clearly know little about Microsoft.

Re: Using AI to match human performance in translating news from Chinese to English

#34

Post by iliketosleep »

londons_explore wrote: post #25
iliketosleep wrote: post #18

I find these types of "match human performance" claims to be ridiculous, especially when it comes to Chinese -> English translations. Translation is both an art and a science, requiring nuanced understanding of the languages, cultures, and context. It also demands quite a bit of creativity. No translation tool I've tried has come even close to matching human performance of a good human translator, including microsoft…

I think there are a lot of bad human translators out there... The number of papers I've read that have very poor grammar, to such an extent it's barely understandable...

I can offer some insight on this, as I've dealt with translations of papers before (Chinese to English). There have been countless times where I've been asked to check the translation of a paper and ended up retranslating it completely. I was often curious as to why the original translation was bad - it was typically either of the following two reasons: 1. The so-called translator was just an overworked research assistant who knows a bit of English. 2. The translation was outsourced to a dodgy company that spends more on marketing than on their translators, usually hiring college grads who did English-related majors.

Re: Using AI to match human performance in translating news from Chinese to English

#35

Post by gumby »

golfer wrote: post #33
anewhnaccount2 wrote: post #3

Compare this to one of Google's blog post promoting their MT research: https://research.googleblog.com/2016/09/a-neural-network-for... It is: 1) More accurate, compared to hyperbole like e.g. "Bridging the Gap between Human and Machine Translation" we have right there in the title the domain: news. 2) A more impressive result. This result is on an independently set up evaluation framework, compared to Google's which…

If you are claiming that Microsoft is pure as the driven snow with regard to making exaggerated and hyperbolic claims, then you clearly know little about Microsoft.

I don't read that claim being made. Microsoft Research has been generally left alone to do good computer science, in which "left alone" has included "by the marketing department"

Re: Using AI to match human performance in translating news from Chinese to English

#36

Post by DonaldFisk »

avgarrison wrote: post #28
vixen99 wrote: post #14

Earlier quoted context omitted.

This was much the point that John Searle made in his criticism of AI prognostications. Ultimately I think he'll be shown to be wrong but the time frame for this will be (I suggest) much longer than is currently touted. https://en.wikipedia.org/wiki/John_Searle

Are you referring to the Chinese Room? I've always had an issue with that argument. Instructions are immutable, but neural networks certainly are not.

Neural networks modify their behaviour by changing their data. Any non-trivial program does that. Some GOFAI programs (e.g. Eurisko) could modify their instructions, not just their data.

Searle's argument is confusing, but how the program in the Chinese Room is implemented doesn't matter. His argument is solely against strong AI. He claims that the Chinese Room (or a suitably programmed computer) cannot be conscious of understanding Chinese in the same way that people can. He doesn't deny that a suitably programmed computer could, in principle, behave as if it understood Chinese, even if it wasn't conscious of anything at all.

However, the machine translation program mentioned in the article behaves as if it understands Chinese only within the limited context of the translation. It wouldn't be able to answer wider questions about things mentioned in an article it had just translated.

Previously it was thought that machine translation systems would have to understand the text they were translating in the way a person does, to produce a useful translation, but that's now shown not to be true. Without hindsight, it's a surprising result, but less surprising when you think of translation as pattern recognition, and think of how a person might go about translating text on a highly technical subject they don't understand.

Re: Using AI to match human performance in translating news from Chinese to English

#37

Post by iliketosleep »

yorwba wrote: post #20
iliketosleep wrote: post #18

I find these types of "match human performance" claims to be ridiculous, especially when it comes to Chinese -> English translations. Translation is both an art and a science, requiring nuanced understanding of the languages, cultures, and context. It also demands quite a bit of creativity. No translation tool I've tried has come even close to matching human performance of a good human translator, including microsoft…

After reading the paper, my takeaway is that humans aren't really very good at translation either. None of the methods scores higher than 70% in the evaluation and that includes several different human translations (whose performance varies greatly depending on how they were sourced). So while matching the quality of the average human translator is a great milestone, there's still lots of room to improve.

Most humans who attempt to translate are not actually translators in the proper sense. Specifically, they are not fully competent in both the source and target language and usually have no formal training in translation. Translation is hard, but there are competent people out there who do it extremely well. Sadly, as an industry it's not taken as seriously as it should be, and most of the people who are actually doing translations do not have the appropriate skill set.

Re: Using AI to match human performance in translating news from Chinese to English

#38

Post by jimbokun »

Quanttek wrote: post #7

Be careful when reading such claims: https://www.theatlantic.com/technology/archive/2018/01/the-s...

That was much better than I expected.

And it wasn't until I looked at the byline at the end when I realized, yes, it is that Hofstadter (Godel, Escher, Bach).

Re: Using AI to match human performance in translating news from Chinese to English

#39

Post by jimbokun »

jakecrouch wrote: post #11

It's obvious that there are limits to how well machine translation can work unless the models have sensory grounding. I wonder if the problem is that people haven't figured out how to do sensory grounding or that the hardware is still too slow for it to work.

This article, posted by Quanttek above, is very relevant to your question:

https://www.theatlantic.com/technology/archive/2018/01/the-s...

Re: Using AI to match human performance in translating news from Chinese to English

#40

Post by Hydraulix989 »

golfer wrote: post #33
anewhnaccount2 wrote: post #3

Compare this to one of Google's blog post promoting their MT research: https://research.googleblog.com/2016/09/a-neural-network-for... It is: 1) More accurate, compared to hyperbole like e.g. "Bridging the Gap between Human and Machine Translation" we have right there in the title the domain: news. 2) A more impressive result. This result is on an independently set up evaluation framework, compared to Google's which…

If you are claiming that Microsoft is pure as the driven snow with regard to making exaggerated and hyperbolic claims, then you clearly know little about Microsoft.

That conclusion doesn't logically follow.
Post reply on HN