Researchers reach human parity in conversational speech recognition

Post by ghurtado » Tue, Oct 18, 2016, 8:11 PM UTC

JoeAltmaier wrote: post #42

ghurtado wrote: post #37

Earlier quoted context omitted.

I can think of no better test of whether some specific being is intelligent than fooling another human into thinking they are. Can you think of a better one? > because it's a behavioral or functional test What else would you test for in an AI other than its output and behavior? Conversely: how would you test a human for intelligence other than through its output and behavior?

Surely that depends upon the human? An infant would do a terrible job. An institutionalized vegetable would fail to provide meaningful information. A not-very-bright cousin of mine has been heard responding to robo-calls. Would his opinion do? I think 'intelligence' is ambiguous, but many parts of it can be measured to some degree. Lets give the AI the SAT test perhaps? Or a test of hypothetical arguments. Or ask it…

Post by Scarblac » Tue, Oct 18, 2016, 8:26 PM UTC

JumpCrisscross wrote: post #22

wbl wrote: post #17

Earlier quoted context omitted.

This is also true for pwople. How do you know humans are thinking and not following lots of social rules they learn thanks to special neural hardware for learning social interaction? Think about the intelligence conveyed in ritualized greetings.

You are recapitulating Descartes. Consciousness appears to be immeasurable. You can only know that you are conscious. It is polite to give others the benefit of doubt.

Post by radarsat1 » Tue, Oct 18, 2016, 8:33 PM UTC

szupie wrote: post #60

radarsat1 wrote: post #35

The term "human parity" refers to a comparison of the error rate, which is a single scalar summarizing performance in terms of mistakes made. It says nothing about the kind of mistakes, and I can easily imagine that machines qualitatively do not make at all the same kind of mistakes as humans. I'd be curious to know if the kind of mistakes machines make might strike human listeners as quite stupid, but maybe not.. ma…

Section 9 in the paper[1] is all about comparing these mistakes between the system and humans. The most common mistakes for humans and the system are in tables 9—11. We find that the artificial errors are substantially the same as human ones with one large exception confusions between backchannel words [acknowledgment words like “uh-huh”] and hesitations . The difference they found, but suspect might be a result of t…

Post by cloudjacker » Tue, Oct 18, 2016, 8:36 PM UTC

joe_the_user wrote: post #59

Eridrus wrote: post #3

The actual paper has a section on error analysis that is particularly enlightening: https://arxiv.org/abs/1610.05256 On the CallHome dataset humans confuse words 4.1% of the time, but delete 6.5% of words, most commonly deleting the word "I". Their ASR system confuses 6.5% of words on this dataset, but only deletes 3.3% of words, so depending on how you view this their claim about being better than humans isn't defin…

I was recently at a bar where they showed a movie with incomprehensible subtitles (English to English). I assume this was because they skimped and bought automatic subtitling. I think one important aspect is while humans miss words, they often get the sentence meaning correct. When computers miss words, they tend to substitute words that sound similar. That's readable if you have time but not necessarily as a stream…

Post by ar15saveslives » Tue, Oct 18, 2016, 8:53 PM UTC

jcoffland wrote: post #27

PravlageTiem wrote: post #20

Earlier quoted context omitted.

A machine process qualifies as AI if it believes in a god. EDIT: I see the nuances of epistemological problems are lost to HN and knee-jerk culture war atheism still rules supreme.

ELIZA could be made to believe in God.