Live data from Hacker News

AI’s Language Problem

technologyreview.com

161–170 of 244 posts

Re: AI’s Language Problem

#161

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

I believe we can get AI from just text. Obviously that won't work for babies, because babies get bored quickly looking at text. AIs can be forced to read billions of words in mere hours! Look at word2vec. By using simple dimensionality reduction on the frequency that words that occur near to each other in news articles, it can learn really interesting things about the meaning of words. The famous example is the vecto…

So a related question becomes, can you learn to understand and thus predict physics (the way a child does - I'm not talking about quantum mechanics) from literature only, without interacting in space?

Re: AI’s Language Problem

#162

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

I believe we can get AI from just text. Obviously that won't work for babies, because babies get bored quickly looking at text. AIs can be forced to read billions of words in mere hours! Look at word2vec. By using simple dimensionality reduction on the frequency that words that occur near to each other in news articles, it can learn really interesting things about the meaning of words. The famous example is the vecto…

> It's learning the meaning of words, and the relationships between them.

It's learning a meaning, not the meaning. It's just a probabilistic model for the occurrence of a word based on the words that surround it. This should not serve as a base for the rest of your claims.

Anyway -- the improvements gained by multi-modal systems essentially disprove your thesis. Which is a good news! We're making progress.

Re: AI’s Language Problem

#163

Earlier quoted context omitted.

I believe we can get AI from just text. Obviously that won't work for babies, because babies get bored quickly looking at text. AIs can be forced to read billions of words in mere hours! Look at word2vec. By using simple dimensionality reduction on the frequency that words that occur near to each other in news articles, it can learn really interesting things about the meaning of words. The famous example is the vecto…

So a related question becomes, can you learn to understand and thus predict physics (the way a child does - I'm not talking about quantum mechanics) from literature only, without interacting in space?

Yes, the child doesn't learn physics he or she learns motor control of its body. Baby learning to talk is more about the brain learning to control its body through throwing its neurological system.

Look at animal kingdom some animals are walking about 5 minutes of being born and running in hours.

Re: AI’s Language Problem

#164
post #142

Earlier quoted context omitted.

It's probably a mistake to assume that just because it's the way we do it that it has to be the way machines do it. Although that's usually the initial assumption. In the early days of flight most attempts were based on birds, similarly submersible vehicles were based on fish. We know now it's better to use propellers. It could be we just haven't found what is analogous to a propeller for the AI world.

The only general intelligence we know of is us. It stands to reason that the first step towards creating AGI is to copy the one machine we know is capable of that type of processing. Why doesn't our research focus on understanding and copying biological brains? Numenta did, with good results, but it isn't an industry trend.

> Why doesn't our research focus on understanding and copying biological brains?

There's lots of basic research being done to better understand the biological brain. Progress is slow and steady, but the brain remains poorly understood at this point in time. Most applied research has pursued more pragmatic methods because these methods have had faster progress and proven more useful in practice.

Re: AI’s Language Problem

#165

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

Here is the reason why I don't believe in that: First of all, it seems that mind space is huge, i.e. there are very different programs that can lead to general intelligence, many of which will be very different from humans (this along is evidenced by the fact how strongly human characters and intellects vary, e.g. highly functioning mental conditions). A lot in machine learning points to that possibility. There are for example many ways to get supervision signals e.g. reconstruction error, prediction error, adversaries, intrinsic motivation (reducing the number of bits required for a representation), compression, sparseness etc.

We basically just need a system that comes up with efficient representations of the world such that it can reason about it, i.e. that it can tell you which hypotheses about the world are likely true given some data. This computation allows you to make predictions and predictions are really at the heart of intelligence. If you can follow a hypothetical trajectory of generated, hallucinated or simulated samples of reality, i.e. samples that likely correspond to what actually happens in the world (and in the agent's own brain), then you can actually perform actions that are targeted at some purpose (e.g. maximizing reward signals). However, there are many sources of data that essentially give you the same information. Whether you create a representation by directly interacting with the world or just watch many examples of how the world generally evolves over time and how different entities interact with one another, you essentially get the same idea about how the world works (except in the first case you also learn a motor system). I think the anthropomorphism is really misplaced here, because computer systems are not dependent on actually performing in the real world. Since computers have near unlimited, noiseless memory and have super fast access to that memory, they can actually plan interactions by careful reasoning on the fly instead of needing to learn e.g. motor skills for manipulating objects, eating and handwriting before one can get anywhere near the performance of computers wrt. access to reliable external memory. A computer system also does not have hormone and neuromodulator levels that need to be met for healthy development (e.g. dopamine), therefore the intuition that deprivation of interaction with the world prevents learning is extremely misleading.

Re: AI’s Language Problem

#166

Earlier quoted context omitted.

> Instead, it is: given a history of moves, can you determine whether or not a computer was playing? No, it's really not. For the Turing Test, the AI is meant to be adversarial—it's objective is to convince you that it is human. AlphaGo's objective isn't to "play like a human," it is to win. If they gave it an objective of playing like a human, I'm sure AlphaGo could play in a way that would be indistinguishable from…

> If they gave it an objective of playing like a human, I'm sure AlphaGo could play in a way that would be indistinguishable from a human. It could just play unbelievably bad and appear like a beginner. That wouldn't prove intelligent. > Peeking at the system/data is cheating Someone ignorant of computers would hardly ever assume a machine. Of course the omission of this rule would leave someone smarter than the comp…

I can only speak for chess, but one of the unsolved problems in computer chess is how to build a program that plays human chess of appropriate level.

It is very hard to dial down a ELO 3000+ program to 1800 level of a club player and not make it computer like.

What is usually done is lower the depth searched and add some random blunders but it is still obvious to a stronger player that it is a program.

Re: AI’s Language Problem

#167
post #99

Earlier quoted context omitted.

That works, if your virtual world is good. But that's a big if. You've just taken one really hard problem (learning about the world) and turned it into an even harder problem (simulating the world).

Some games are pretty convincing sand boxes and it's not obvious to me that you really need full range of senses to properly teach the AI.

[deleted]

Re: AI’s Language Problem

#168

Earlier quoted context omitted.

> I don't find the Turing Test convincing either, because someone smart enough to build it should be smart enough to recognize it from its answers. Why do you assume that? The creators of AlphaGo certainly couldn't beat it.

Good question. Someone beat it. He and his games as training sets were part of the development of AlphaGo development. I edited the post, did you read that? You are making my point, you can't bootstrap a definition for artificial intelligence by comparison to humans, when human intelligence is not well defined either.

I read your post, but it's very muddled. You might consider advancing a clearer thesis, because it seems that you are under the impression that it's impossible for humans to build systems which are smarter than themselves.

The first versions of AlphaGo were certainly inferior to human players, but the current version is superior to any human.

Re: AI’s Language Problem

#169

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

> you end up with something which is not particularly different or better than a human cyborg.

Even if that were true, which I am not sure about, such a robot will have a very different moral status, as an artifact, and one that can be reproduced cheaply and indefinitely. This is very useful indeed, so on this axis it could be counted as 'better'.

The idea of development being important for AI is an old one, but it hasn't had much concrete success. Brook's Cog robot at MIT is a prominent example of a robot that didn't do very much, despite this approach being taken in a good faith effort by talented and well-supported people.

Re: AI’s Language Problem

#170

Earlier quoted context omitted.

> Instead, it is: given a history of moves, can you determine whether or not a computer was playing? No, it's really not. For the Turing Test, the AI is meant to be adversarial—it's objective is to convince you that it is human. AlphaGo's objective isn't to "play like a human," it is to win. If they gave it an objective of playing like a human, I'm sure AlphaGo could play in a way that would be indistinguishable from…

> If they gave it an objective of playing like a human, I'm sure AlphaGo could play in a way that would be indistinguishable from a human. It could just play unbelievably bad and appear like a beginner. That wouldn't prove intelligent. > Peeking at the system/data is cheating Someone ignorant of computers would hardly ever assume a machine. Of course the omission of this rule would leave someone smarter than the comp…

> It could just play unbelievably bad and appear like a beginner. That wouldn't prove intelligent.

Sure, which is why it's not a very good metric. The correct metric for looking at whether computational game intelligence has exceeded human capacity is that computers can consistently beat humans.

To be clear, I'm not convinced that we'll ever make a generalized intelligence which can pass the Turing Test. My point was merely that the fact that humans create the system is not a good argument for why it's impossible: in many domains, we can already create computer systems which vastly outperform ourselves.

Post reply on HN