AI’s Language Problem • Hacker News Board

Re: AI’s Language Problem

#101

Post by rbobby » Tue, Aug 09, 2016, 6:13 PM UTC

erikpukinskis wrote: post #84

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

> locking a baby in a featureless room

Strangely (or maybe horribly?) there are a number of studies of children raised in Romanian orphanages that somewhat cover this area.

Under Nicolae Ceaușescu the government outlawed abortion (with some exceptions) in an attempt to increase the birth rate (https://en.wikipedia.org/wiki/Abortion_in_Romania). Coupled with a poor economy this lead to masses of infants and children being given over to government "care" in orphanages. These were pretty bleak places for infants and children, with infants often spending hours in a crib with little stimulation.

Here's a really good article about the effects this sort of institutional "care" has on children: http://www.americanscientist.org/issues/feature/2009/3/the-d...

Here's the The Bucharest Early Intervention Project (tons of info about this subject area): http://www.bucharestearlyinterventionproject.org/

Re: AI’s Language Problem

#102

Post by Houshalter » Tue, Aug 09, 2016, 6:14 PM UTC

erikpukinskis wrote: post #84

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

I believe we can get AI from just text. Obviously that won't work for babies, because babies get bored quickly looking at text. AIs can be forced to read billions of words in mere hours!

Look at word2vec. By using simple dimensionality reduction on the frequency that words that occur near to each other in news articles, it can learn really interesting things about the meaning of words. The famous example is the vector "king" minus the vector for "man" plus "woman" equals "queen". It's learning the meaning of words, and the relationships between them.

Recurrent NNs, using similar techniques, but with much more complexity, can learn to predict the next word in a sentence very well. They can learn to write responses that are almost indistinguishable from humans. And it's incredible this works at all, given RNNs have only a few thousand neurons at most, and a few days of training, compared to humans' billions of neurons trained over a lifetime.

All of the information of our world is contained in text. Humans have produced billions of books, papers, articles, and internet comments. Billions of times more information than any human could read in their entire lifetime. Any information you can imagine is contained in text somewhere. I don't think it's necessary for AIs to be able to see, or interact with the world in any way.

If you can predict the word a human would say next, with enough accuracy, then you could also produce answers indistinguishable from theirs. Meaning you could pass the Turing test, and perform any language task they could do just as well. So language prediction alone may be sufficient for AGI.

This is the theory behind the Hutter Prize, which proposes that predicting (compressing) wikipedia's text is a measure of AI progress. The Hutter Prize isn't perfect (it's only a sample of wikipedia, which is very small compared to all the text humans have produced), but the idea is solid.

Re: AI’s Language Problem

#103

Post by scotty79 » Tue, Aug 09, 2016, 6:19 PM UTC

Language is just a protocol for synchronizing slices of two world models, the one in the head of the one who talks and the one in the head of who listens. If the recipient doesn't have the model similar to the one sender has, language is meaningless.

You can't "understand" language without having model of the world that humans construct during their life and education.

So pretty much next step for language recognition is indistinguishable from sentience.

Re: AI’s Language Problem

#104

Post by posterboy » Tue, Aug 09, 2016, 6:20 PM UTC

protomikron wrote: post #90

erikpukinskis wrote: post #84

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

> This isn't an engineering problem, it's a philosophical problem [...] Indeed. However we can not rule out the possibility that we engineer a system without a "body" (I think you are referring to the Embodied mind thesis?). It is a complicated topic and discussions about it are futile without precise definitions of loaded terms, like "intelligence", "body", etc. A rather well-defined test is the classic Turing test…

I don't find the Turing Test convincing either, because someone smart enough to build it should be smart enough to recognize it from its answers. And if that depends on the intelligence of the questioner, whose intelligence is tested then, really?

IIRC the test is a binary classifier, but intelligence is a spectrum that's fuzzy and therefore inherently hard to define.

IE, how low is someone willing to assume the lack of intelligence in a human is not a good definition of general intelligence, as that's circular reasoning.

I would suppose that babies possess general intelligence, but they lack the knowledge about the environment.

Re: AI’s Language Problem

#105

Post by ygmelnikova » Tue, Aug 09, 2016, 6:27 PM UTC

Seems about every decade there's a resurgence of interest in AI and 'chat bots' that always ends with the same conclusion. I wrote a similar article over 20 years ago titled 'Why the Loebner prize will never be won'. There's a great quote; 'The real technology, behind all the other technology, is language'. Unless this can get figured out, artificial sentient beings with capabilities like those found in sci-fi will remain wishful thinking, and that isn't happening anytime soon.

Let's come back and revisit this in 10 years.

Re: AI’s Language Problem

#106

Post by wrongc0ntinent » Tue, Aug 09, 2016, 6:28 PM UTC

erikpukinskis wrote: post #84

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

In fact it is a very hard engineering problem, which doesn't exclude a "philosophical" one. There's a line of research surrounding anthropomimetic robots with the specific task of studying cognition, going back at least a decade.

Re: AI’s Language Problem

#107

Post by YeGoblynQueenne » Tue, Aug 09, 2016, 6:33 PM UTC

The problem with natural language processing is that we are trying to learn it (read: construct models of it) from utterances, things that are being said or transcribed. And that is a big, huge problem because there is a lot more to language than utterances. Hell, there is a lot more to language than language itself.

There are things you cannot put into words, and yet you think them. There are things that you can't put into words and yet you can make people around you understand them. There are things you understand without even knowing you understand them. But even before we go there- there are so many things that people can make utterances about that are not possible to collect into example sets and train models on.

How do you collect examples of whatever it is that makes people lie on the beach to get a sun tan? How do you collect examples of imagination, dreams, abstract thinking, all those things that your brain does that may be a side-effect of self-aware intelligence or the whole point of self-aware intelligence in the first place?

How do you collect a data set that's as big as the whole world you've experienced in your however many years of life? And even if you could, what machine has the processing power to train on that stuff, again and again, until it gets it right?

Machine learning meaning is hopeless, folks. Fuggeddabout it. There's not enough data in the whole world, there's no machine big enough to process it if it existed. We 'll make some advances in text processing, sure, we'll automate some useful stuff like translation (for languages close to each other) and captioning (for photographs) and then we'll stall until the next big thing comes about in a few generations from now.

That's what the current state of the art suggests.

Re: AI’s Language Problem

#108

Post by justinpombrio » Tue, Aug 09, 2016, 6:36 PM UTC

tener wrote: post #92

erikpukinskis wrote: post #84

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

Why not simply simulate the body (or indeed multiple bodies) in a virtual world? Easier to build, parallelize, extend, maintain. Possibly somewhat safer too.

This is exactly the premise of The Talos Principle :-)

Re: AI’s Language Problem

#109

Post by posterboy » Tue, Aug 09, 2016, 6:37 PM UTC

erikpukinskis wrote: post #84

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

Computers are programmed only using text, even if the text has just two symbols. Sensor interfaces use digital signals, again symbol streams.

That would beget the question if there could be machines mightier than a Turing Complete one. I'm sure that's missing your point.

Re: AI’s Language Problem

#110

Post by protomyth » Tue, Aug 09, 2016, 6:41 PM UTC

Has anyone used something like an MMO to get people to help train their AI in an artificial environment?