AI’s Language Problem • Hacker News Board

Re: AI’s Language Problem

#201

Post by Houshalter » Wed, Aug 10, 2016, 1:48 AM UTC

ACow_Adonis wrote: post #188

Houshalter wrote: post #102

Earlier quoted context omitted.

I believe we can get AI from just text. Obviously that won't work for babies, because babies get bored quickly looking at text. AIs can be forced to read billions of words in mere hours! Look at word2vec. By using simple dimensionality reduction on the frequency that words that occur near to each other in news articles, it can learn really interesting things about the meaning of words. The famous example is the vecto…

> All of the information of our world is contained in text Communicate the concept of "green" to me in text. The sound of a dog barking, a motor turning over, a sonic boom, or the experience of a Doppler shift. Beethoven's symphony. Sour. Sweet. What does "mint" taste like? Shame. Merit. Learn facial and object recognition via text. Vertigo. Tell a boxer how to box by reading? Hand eye coordination, bodies in 3 dimen…

Are blind or deaf people not intelligent?

But if you must pretend to be sighted and hearing, there are many descriptions of green, of dogs barking, of motors, etc, scattered through the many books written in English (and other languages.)

Are these descriptions perfect? Maybe not. But they are sufficient to mimic or communicate with humans through text. It's sufficient to beat a Turing test, to answer questions intelligently, to write books and novels, and political arguments, etc. If that's not AGI, I don't know what is.

Re: AI’s Language Problem

#202

Post by MichaelMoser123 » Wed, Aug 10, 2016, 1:48 AM UTC

Houshalter wrote: post #102

erikpukinskis wrote: post #84

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

I believe we can get AI from just text. Obviously that won't work for babies, because babies get bored quickly looking at text. AIs can be forced to read billions of words in mere hours! Look at word2vec. By using simple dimensionality reduction on the frequency that words that occur near to each other in news articles, it can learn really interesting things about the meaning of words. The famous example is the vecto…

> Look at word2vec. By using simple dimensionality reduction on the frequency that words that occur near to each other in news articles, it can learn really interesting things about the meaning of words

doesn't that depend on the syntax of a language? my guess is that it would not work for a language that has a more flexible word order than English; so it wouldn't quite work for such languages as Russian, Turkish and Finnish (and other languages).

Re: AI’s Language Problem

#203

Post by jonkho » Wed, Aug 10, 2016, 1:49 AM UTC

Houshalter wrote: post #193

jonkho wrote: post #190

Earlier quoted context omitted.

I second this. To further illustrate that semantics is far more than syntax and that emotions are an inseparable part of understanding language, do the following experiment: take any word or concept and look up its definition in a dictionary. Take note of the words used in the definition and look up the meaning of those words. Repeat recursively and if you go far enough down the tree you will find that all words lead…

What does the experiment prove? Of course dictionaries are circular. But that doesn't mean they don't contain huge amounts of information about the world, and about language. Information an AI could infer without any interaction with the outside world at all.

This means that the semantics of the language is rooted in subjective states. Restated, this means humans "understand" language because of humans' emotions. Computers may "understand" language too, but it surely will not be due to the subjective states as it is with humans. If we define AI as a computer that must "understand" language the same way as humans do, then by definition, AI is not possible.

Re: AI’s Language Problem

#204

Post by Natanael_L » Wed, Aug 10, 2016, 1:54 AM UTC

Houshalter wrote: post #151

jacobrobbins wrote: post #137

Earlier quoted context omitted.

> All of the information of our world is contained in text. This statement is false. There is a well known thought experiment called Mary’s Room the gist of which is that knowing all conceivable scientific knowledge about how humans perceive color is still not a substitute for being a human and perceiving the color red: https://philosophynow.org/issues/99/What_Did_Mary_Know The experience of seeing red is an example…

Mary's Room and qualia is totally irrelevant. I'm not asking if the computer will "feel" "redness", simply if it can pretend to do so through text. If it can talk about the color red, in a way indistinguishable from any other human talking about red. In any case, at some level everything is symbols. A video is just a bunch of 1's and 0's, as is text, and everything else. A being raised on only text input would have q…

Like explained elsewhere here, it may be that our brains share a genetically coded "decryption key", and that many of the things we talk about are too poorly and noisily expressed for a purely text based computer AI to ever truly replicate the processes going on inside our brains. Insufficient data, simply put, and no way to get it.

It may sure look indistinguishable, but on the inside it just wouldn't be the same.

Re: AI’s Language Problem

#205

Post by Houshalter » Wed, Aug 10, 2016, 2:06 AM UTC

jonkho wrote: post #203

Houshalter wrote: post #193

Earlier quoted context omitted.

What does the experiment prove? Of course dictionaries are circular. But that doesn't mean they don't contain huge amounts of information about the world, and about language. Information an AI could infer without any interaction with the outside world at all.

This means that the semantics of the language is rooted in subjective states. Restated, this means humans "understand" language because of humans' emotions. Computers may "understand" language too, but it surely will not be due to the subjective states as it is with humans. If we define AI as a computer that must "understand" language the same way as humans do, then by definition, AI is not possible.

This is why Turing invented his famous test. At the time people were arguing about what it would take to prove a machine is intelligent. People argued that the internal properties of the machine mattered, that it needed to feel emotions and qualia just like humans.

But Turing argued that if the machine could be shown to do the same tasks that humans can do, and act indistinguishable from a real human, then surely it's intelligent. Its internal properties don't matter, at least not to judge intelligence.

Re: AI’s Language Problem

#206

Post by jonkho » Wed, Aug 10, 2016, 2:33 AM UTC

Houshalter wrote: post #205

jonkho wrote: post #203

Earlier quoted context omitted.

This means that the semantics of the language is rooted in subjective states. Restated, this means humans "understand" language because of humans' emotions. Computers may "understand" language too, but it surely will not be due to the subjective states as it is with humans. If we define AI as a computer that must "understand" language the same way as humans do, then by definition, AI is not possible.

This is why Turing invented his famous test. At the time people were arguing about what it would take to prove a machine is intelligent. People argued that the internal properties of the machine mattered, that it needed to feel emotions and qualia just like humans. But Turing argued that if the machine could be shown to do the same tasks that humans can do, and act indistinguishable from a real human, then surely it'…

Exactly. The computer will fail such a test where such tests can only be passed if and only if the agent under test experiences subjective states. Since humans can always craft this class of tests, and the computer cannot pass it, it will always fail the "Turing Test".

Re: AI’s Language Problem

#207

Post by sullyj3 » Wed, Aug 10, 2016, 2:37 AM UTC

erikpukinskis wrote: post #84

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

This sort of anthropomorphic intuition pump is counter production. It would be much easier to believe that a baby could do that if we designed the baby's brain from scratch to be able to.

Re: AI’s Language Problem

#208

Post by sullyj3 » Wed, Aug 10, 2016, 2:40 AM UTC

ACow_Adonis wrote: post #188

Houshalter wrote: post #102

Earlier quoted context omitted.

I believe we can get AI from just text. Obviously that won't work for babies, because babies get bored quickly looking at text. AIs can be forced to read billions of words in mere hours! Look at word2vec. By using simple dimensionality reduction on the frequency that words that occur near to each other in news articles, it can learn really interesting things about the meaning of words. The famous example is the vecto…

> All of the information of our world is contained in text Communicate the concept of "green" to me in text. The sound of a dog barking, a motor turning over, a sonic boom, or the experience of a Doppler shift. Beethoven's symphony. Sour. Sweet. What does "mint" taste like? Shame. Merit. Learn facial and object recognition via text. Vertigo. Tell a boxer how to box by reading? Hand eye coordination, bodies in 3 dimen…

I don't think raw feels, qualia type stuff really counts as information in the information theoretic sense. Nor is understanding its nature necessary for artificial general intelligence (though perhaps it is (or perhaps not) for artificial consciousness, which is not the same thing.)

Re: AI’s Language Problem

#209

Post by sullyj3 » Wed, Aug 10, 2016, 2:42 AM UTC

ACow_Adonis wrote: post #188

Houshalter wrote: post #102

Earlier quoted context omitted.

I believe we can get AI from just text. Obviously that won't work for babies, because babies get bored quickly looking at text. AIs can be forced to read billions of words in mere hours! Look at word2vec. By using simple dimensionality reduction on the frequency that words that occur near to each other in news articles, it can learn really interesting things about the meaning of words. The famous example is the vecto…

> All of the information of our world is contained in text Communicate the concept of "green" to me in text. The sound of a dog barking, a motor turning over, a sonic boom, or the experience of a Doppler shift. Beethoven's symphony. Sour. Sweet. What does "mint" taste like? Shame. Merit. Learn facial and object recognition via text. Vertigo. Tell a boxer how to box by reading? Hand eye coordination, bodies in 3 dimen…

One way of looking at it is that some robots know how to walk, and this knowledge is encoded as a string of ones and zeros on a storage medium.

Re: AI’s Language Problem

#210

Post by dmreedy » Wed, Aug 10, 2016, 2:51 AM UTC

eli_gottlieb wrote: post #194

dmreedy wrote: post #134

Earlier quoted context omitted.

It can be the case if Chomsky was right, and Universal Grammar and other similar structures are a thing. That would mean that part of our ability to understand language comes from the particular structure of our brain (which everyone seems to by and large share). That would mean that some of our ability to understand language is genetic in nature, by whatever means genes direct the structure of brain development.

Nah, it's mostly just hierarchical probability models. http://www.ncbi.nlm.nih.gov/pubmed/24977647

This is a very neat paper, but:

1) They appear to have crafted the skeleton of a grammar as it is with their nodes, super nodes, and slot collocations. This is directly analogous to something like an Xbar grammar, and is not learned by the system; therefore, if anything, it's strengthening a Chomskian position; the system is learning how a certain set of signals satisfy its extant constraints.

2) The don't appear to go beyond generative grammar, which already seems largely solvable by other ML methods, and is a subset of the problem "language". Correct me if I'm wrong here, it's a very long paper and I may have missed something.