Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI’s Language Problem

technologyreview.com

Post reply on HN
Print view View original

171–180 of 244 posts

Re: AI’s Language Problem

#171

Post by gluczywo »

AI researchers overestimate the role of language in the development of human-like intelligence understood as a common sense heuristic physics that is built empirically through non-linguistic experiences (e.g. if I turn that glass, the wine spills and leaves dirty stain on the couch what has further consequences). The one that is most difficult to implement/reproduce/emulate by machine.

Language is only a communication protocol most efficient in an interactive context (dialog) that allows two agents with shared but not identical set of experiences achieve understanding in some domain and context with the caveat that understanding is unprovable and not absolute. Understanding that is only empirically tested and behavior probed (e.g. long after successful conversation Agent Alice discovers that Agent Bob "did not get it" as she expected).

By analyzing sole language without experiences, the machine, using something like word2vec, may discover semantic dependencies (e.g. man + cassock = pedophile) but not true semantics that has world consequences.

Even with unlimited language corpora the machine does not have the set of axioms that humans have (experiences and observed stories). These axioms are needed to build further more abstract knowledge.

Re: AI’s Language Problem

#172

Post by robotresearcher »

protomikron wrote: post #90
erikpukinskis wrote: post #84

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

> This isn't an engineering problem, it's a philosophical problem [...] Indeed. However we can not rule out the possibility that we engineer a system without a "body" (I think you are referring to the Embodied mind thesis?). It is a complicated topic and discussions about it are futile without precise definitions of loaded terms, like "intelligence", "body", etc. A rather well-defined test is the classic Turing test…

> It is a complicated topic and discussions about it are futile without precise definitions of loaded terms, like "intelligence", "body", etc.

On the contrary, to make progress we should give up on making precise definitions of non-technical terms such as 'body' and 'intelligence'. They are folk notions that don't have relevant precise definitions talking about them in an engineering context is distracting.

You don't need a definition of 'beauty' to paint. You don't need a definition of 'justice' to practice law. You don't need a definition of 'intelligence' to build clever robots.

Re: AI’s Language Problem

#173

Post by [deleted] »

bweitzman wrote: post #153
jacobrobbins wrote: post #137

Earlier quoted context omitted.

> All of the information of our world is contained in text. This statement is false. There is a well known thought experiment called Mary’s Room the gist of which is that knowing all conceivable scientific knowledge about how humans perceive color is still not a substitute for being a human and perceiving the color red: https://philosophynow.org/issues/99/What_Did_Mary_Know The experience of seeing red is an example…

The Mary's Room thought experiment is garbage, if you ask me. You can't just assume your hypothesis and then call the result truth. If you assert that a person can understand everything there is to know about the color red and then still not understand what it is like to see red, you have either contradicted yourself or assumed dualism.

[deleted]

Re: AI’s Language Problem

#174

Post by orthoganol »

Houshalter wrote: post #154
orthoganol wrote: post #138

Earlier quoted context omitted.

I think if AGI were possible from the basic statistical NLP techniques outlined in most advanced NLP textbooks, it would have already happened a decade ago.

I'm not saying it is possible from just basic statistical NLP techniques. It may take much more advanced techniques. And it may take much more computing power than we have even now. But I do believe it is possible, someday. Probably within our lifetime.

I certainly think AGI is possible. I just don't think Word2vec, RNN's, i.e. stuff from the NLP textbooks, is in the same ballpark as what it will take to achieve.

Edit - To be more clear I also agree AGI could be possible with text as the only input. I just think we need a new paradigm. Ostensibly AGI is meant to mimic human intelligence (minus the pitfalls), so IMO, the best approach will be that which mimics the underlying processes of human intelligence - not just the results. Traditional statistical NLP methods will probably have some role in this final system, but not the heart of it, as far as mimicking intelligence by mimicking intelligence's underlying processes goes.

Re: AI’s Language Problem

#175

Post by dragonwriter »

erikpukinskis wrote: post #84

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

> They need bodies that are biologically connected with the rest of the biosphere

On what basis do you conclude that "biologically" is important here? There may be some reason to suspect that human-like intelligence requires human-like ability to sense and interact with the outside world, but I see less reason to suspect that it is important that the mechanism of the sensors or manipulators must be biological.

Re: AI’s Language Problem

#176

Post by xapata »

wlievens wrote: post #161
Houshalter wrote: post #102

Earlier quoted context omitted.

I believe we can get AI from just text. Obviously that won't work for babies, because babies get bored quickly looking at text. AIs can be forced to read billions of words in mere hours! Look at word2vec. By using simple dimensionality reduction on the frequency that words that occur near to each other in news articles, it can learn really interesting things about the meaning of words. The famous example is the vecto…

So a related question becomes, can you learn to understand and thus predict physics (the way a child does - I'm not talking about quantum mechanics) from literature only, without interacting in space?

> from literature only

How about when you include multimedia recordings? Or give the machine a camera and wheels?

Re: AI’s Language Problem

#177

Post by hacker42 »

hacker42 wrote: post #127
Houshalter wrote: post #102

Earlier quoted context omitted.

I believe we can get AI from just text. Obviously that won't work for babies, because babies get bored quickly looking at text. AIs can be forced to read billions of words in mere hours! Look at word2vec. By using simple dimensionality reduction on the frequency that words that occur near to each other in news articles, it can learn really interesting things about the meaning of words. The famous example is the vecto…

I agree that learning to reason about the world likely does not require experience with motor control and proprioception (i.e. literally how babies do it), though I do think that you either need at least some sort of tempo-spatial experience (e.g. visual). Tempo-spatial representations are just extremely hard to convey by text only. You might get the idea of closeness by saying 'close is when two words are close in a…

tl;dr I think you need a small set of tempo-spatially grounded meanings (though not necessarily agent-related) and you can bootstrap everything from that using only textual knowledge.

Re: AI’s Language Problem

#178

Post by hyperpallium »

> his team was just as surprised as everyone else ... It was only several days later, after careful analysis, that the Google team made a discovery

Since success has a higher priority for researchers than explicable success, and if the "singularity" is just progress that is not understood, it may be almost here - and not require true AI.

Though to be fair, by that definition, the singularity has always been with us, since we don't understand how we think.

Re: AI’s Language Problem

#179

Post by wangchow »

robotresearcher wrote: post #169
erikpukinskis wrote: post #84

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

> you end up with something which is not particularly different or better than a human cyborg. Even if that were true, which I am not sure about, such a robot will have a very different moral status, as an artifact, and one that can be reproduced cheaply and indefinitely. This is very useful indeed, so on this axis it could be counted as 'better'. The idea of development being important for AI is an old one, but it h…

Humans seem to be created cheaply and indefinitely too. Population is growing out of control. We could attach them to pods and harvest energy from their souls!

Re: AI’s Language Problem

#180

Post by DonaldFisk »

Houshalter wrote: post #133
dmreedy wrote: post #130

Earlier quoted context omitted.

> All of the information of our world is contained in text Even if this were a true statement, it's still the case that it might not be enough. There is a class of functions that are simply not learnable without some prerequisite knowledge. This is directly analogous to a one-time pad in crypto. It is entirely possible that the function 'language' is in this class of unlearnable functions. While it may be the case th…

>There is a class of functions that are simply not learnable without some prerequisite knowledge. This is directly analogous to a one-time pad in crypto. It is entirely possible that the function 'language' is in this class of unlearnable functions. I don't understand how this could possibly be the case. We can already make great progress on language understanding with simple methods like word2vec, or perhaps even ma…

Wouldn't you still need to attach meanings to the words though? How could an AI system ever understand, for example, the Voynich Manuscript? There's plenty of text in it, and encryption methods when it was written weren't particularly strong. Or how would a person do if they were locked in a room with lots of books written in a language unknown to them?
Post reply on HN