Live data from Hacker News

Using AI to match human performance in translating news from Chinese to English

blogs.microsoft.com

51–60 of 62 posts

Re: Using AI to match human performance in translating news from Chinese to English

#51
post #30

Earlier quoted context omitted.

Let's just define that "human like" in context of machine translation from now on mean "with full legal responsibility". Then let's see who claim their translator is "human like".

Well, Hiroshima and Nagasaki bombing was allegedly due to a translation error. It will certainly take time for a machine translation to catch up.

Can you provide a source with more information about that, please?

Re: Using AI to match human performance in translating news from Chinese to English

#53
post #51

Earlier quoted context omitted.

Well, Hiroshima and Nagasaki bombing was allegedly due to a translation error. It will certainly take time for a machine translation to catch up.

Can you provide a source with more information about that, please?

https://www.nytimes.com/1989/08/21/opinion/l-good-translatio...

Apparently the translation of the Japanese response to the Allies ultimatum calling for their surrender might have been faulty.

Re: Using AI to match human performance in translating news from Chinese to English

#55
post #51

Earlier quoted context omitted.

Well, Hiroshima and Nagasaki bombing was allegedly due to a translation error. It will certainly take time for a machine translation to catch up.

Can you provide a source with more information about that, please?

New Approach to Legal Translation By Susan Šarčević ISBN 9041104011 chapter 7.4 page 201

Re: Using AI to match human performance in translating news from Chinese to English

#56

Compare this to one of Google's blog post promoting their MT research: https://research.googleblog.com/2016/09/a-neural-network-for... It is: 1) More accurate, compared to hyperbole like e.g. "Bridging the Gap between Human and Machine Translation" we have right there in the title the domain: news. 2) A more impressive result. This result is on an independently set up evaluation framework, compared to Google's which…

I haven't looked at this work, but I found the write up of this work pretty misleading: https://www.microsoft.com/en-us/research/blog/microsoft-rese...

The issue with that evaluation was that machines were much better at not making some trivial mistakes humans don't care about (eg transcribing umm, err, etc), but were more likely to get the meaning wrong. Kudos to MS for doing the error analysis and publishing that info, but I found the reporting of it misleading.

Re: Using AI to match human performance in translating news from Chinese to English

#57

Earlier quoted context omitted.

Are you referring to the Chinese Room? I've always had an issue with that argument. Instructions are immutable, but neural networks certainly are not.

Neural networks modify their behaviour by changing their data. Any non-trivial program does that. Some GOFAI programs (e.g. Eurisko) could modify their instructions , not just their data. Searle's argument is confusing, but how the program in the Chinese Room is implemented doesn't matter. His argument is solely against strong AI. He claims that the Chinese Room (or a suitably programmed computer) cannot be conscious…

It's a completely meaningless and plain stupid argument though. There's no reason to believe human consciousness is special and that "understanding" Chinese is at all related to it. One can speculate that the perception of consciousness is simply the product of some form of introspective sensory system and internally-directed actions, of which the former ironically clearly doesn't quite work when it comes to language (or it wouldn't be as much of a reverse-engineering exercise). Nothing really stops you from throwing that into your system and having it consciously understand chinese, assuming it already maintains state.

You can however make a fairly solid argument that a CNN alone (as used in image/object recognition) is fundamentally incapable of dealing with images (but maybe not language), on the assumption that it can be faithfully described as Satan's boolean satisfiability problem, then by virtue of complexity theory it can only be solved in constant time with a sufficiently massive lookup table (which there wouldn't be enough atoms in the universe to store). Microsoft are actually dealing with this in their system by repeatedly applying the network and revisioning the text.

Regardless though, accurate NLP is going to come down to managing to codify how humans deals with objects, concepts and actions, because that's what the languages encode; GOFAI wasn't really too off (and the original effort was doomed from the start by the state of hardware and linguistics). Consider how distinguishing objects as masculine-feminine-neuter and animate-inanimate(-human) is universal (but doesn't necessarily affect the grammar), and that the latter is based purely on how complex/incomprehensible the behaviour of something is (unlike grammatical gender which seems to be fairly arbitrary). Of course that's arguable, but you can see animacy appear in english word choices (unrelated to anthromorphic metaphors) and in how "animate" objects tend to be referred to as having intent. You could try and figure all this out the wrong way around using statistical brute force and copious amounts of text, but that's pretty roundabout isn't it?

(Also, the assumption that an objects animacy is determined by predictability offers a pretty concise explanation of why the idea of human consciousness being produced by simpl(er) interacting systems often fails to compute so spectacularily, why most programmers appear to be immune to that, and also why the illusion that image recognition CNNs perform their intended function is so strong (regardless of how useful they are, the failures make it blatant that they're only looking at texture and low-level features, and are extremely sensitive to noise, which is the opposite of what anyone intended))

Re: Using AI to match human performance in translating news from Chinese to English

#58

I find these types of "match human performance" claims to be ridiculous, especially when it comes to Chinese -> English translations. Translation is both an art and a science, requiring nuanced understanding of the languages, cultures, and context. It also demands quite a bit of creativity. No translation tool I've tried has come even close to matching human performance of a good human translator, including microsoft…

Some people without a specific cultural understanding happen to translate the content perfectly well, too. Experience and constant mastering of translational skills is what counts. If experienced and skilled translators are cooperating in developing process, it could be more than possible to get to a human performance.

Re: Using AI to match human performance in translating news from Chinese to English

#59
post #49
post #44

Earlier quoted context omitted.

That's not his argument at all. His argument is that just because you can do some task doesn't mean you "understand" it. You don't need some fancy philosophy and complex thought experiments to see what he means. Just look at how people learn math. You can do calculations by memorizing algebraic rules, but that's not the same as understanding why those rules exist and what they mean . Even though you will calculate an…

Thanks for rebutting Chathamization's gross misrepresentation of Searle's argument. > It's all theoretical as long as you stick to simple tests (like multiple-choice exams), but becomes relevant when you suddenly expand the context (like requiring the student to prove some theorem instead of doing a calculation). Not even expanding the context changes the situation. The proof of a theorem can be memorized without any…

> The proof of a theorem can be memorized without any understanding just as easily as algebraic rules.

Sure, but in practice students who rely purely on memorization can't answer questions that go beyond what's directly covered in textbooks.

Re: Using AI to match human performance in translating news from Chinese to English

#60
post #29
post #6

The most impressive ML translation tool I've seen so far is DeepL[0]. Sometimes, it manages to translate whole articles without errors. [0]: https://www.deepl.com/translator

Impressive, but there's an easy formula for getting these systems to make mistakes. Just input a sentence with some kind of long distance dependency. For example, DeepL gets agreement right in English to Spanish translations when the two things that agree are close together: I like soup -> COMO sopa They eat soup -> COMEN sopa Impressively, it can even get agreement correct across clause boundaries in many cases. But…

[deleted]
Post reply on HN