Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI’s Language Problem

technologyreview.com

Post reply on HN
Print view View original

221–230 of 244 posts

Re: AI’s Language Problem

#221

Post by Houshalter »

argonaut wrote: post #220
Houshalter wrote: post #201

Earlier quoted context omitted.

Are blind or deaf people not intelligent? But if you must pretend to be sighted and hearing, there are many descriptions of green, of dogs barking, of motors, etc, scattered through the many books written in English (and other languages.) Are these descriptions perfect? Maybe not. But they are sufficient to mimic or communicate with humans through text. It's sufficient to beat a Turing test, to answer questions intel…

Yes they are. However, is a blind, deaf, person with absolutely no motor control, no sense of touch, and no proprioception intelligent? Unclear. They certainly have no language faculties.

But a blind person can't describe green. A deaf person can't describe the sound of a motorboat. A person without taste can't describe mint flavor. That is the point I was making.

I don't propose that a human could lose all of their senses and still be able to communicate. But I do believe computers could do so, if they are designed to do that. Humans are not designed to work lacking those senses.

Re: AI’s Language Problem

#222

Post by argonaut »

Houshalter wrote: post #221
argonaut wrote: post #220

Earlier quoted context omitted.

Yes they are. However, is a blind, deaf, person with absolutely no motor control, no sense of touch, and no proprioception intelligent? Unclear. They certainly have no language faculties.

But a blind person can't describe green. A deaf person can't describe the sound of a motorboat. A person without taste can't describe mint flavor. That is the point I was making. I don't propose that a human could lose all of their senses and still be able to communicate. But I do believe computers could do so, if they are designed to do that. Humans are not designed to work lacking those senses.

So a blind person would never be able to understand the different categories of color (other than that they are placeholders for distinct categories of something).

Now we are just speculating. We believe a computer might be able to understand things for which it doesn't have the sense - but that is speculation and totally untested, and certainly can no longer be justified by using human minds as an example.

Re: AI’s Language Problem

#223

Post by visarga »

dmreedy wrote: post #66
visarga wrote: post #51

Earlier quoted context omitted.

Grounding language in other sense modalities (multimodal learning) is a thing. We can even generate captions from images and generate images from captions, albeit, not perfectly. Another grounding source is related to ontologies. We are already building huge maps of facts about the world like "object1 relation object2". Another source of "common sense" is word embeddings. In fact it is possible to embed all kinds of…

image caption is a separate, albeit related problem to what I'm talking about. Ontologies are much the same; they are interesting for the problems they solve, but it's not clear how well those problems relate to the more general problem of language. word embeddings are also quite interesting, but again, are typically based entirely off whatever emergent semantics can be gleaned from the structure of documents. It's n…

I think the key ingredient is to be reinforcement learning, and more importantly, agents being embedded in the external world.

Regarding the "internal world", we already see the development of AI mechanisms for attention, short term memory (references to concepts recently used), episodic memory (autobiographic) and semantic memory (ontologies).

Re: AI’s Language Problem

#224

Post by posterboy »

morgante wrote: post #168
posterboy wrote: post #136

Earlier quoted context omitted.

Good question. Someone beat it. He and his games as training sets were part of the development of AlphaGo development. I edited the post, did you read that? You are making my point, you can't bootstrap a definition for artificial intelligence by comparison to humans, when human intelligence is not well defined either.

I read your post, but it's very muddled. You might consider advancing a clearer thesis, because it seems that you are under the impression that it's impossible for humans to build systems which are smarter than themselves. The first versions of AlphaGo were certainly inferior to human players, but the current version is superior to any human.

> because it seems that you are under the impression

I made a hopeful hypothesis and I opposed immediately that human intelligence might just not be optimized for recognizing intelligence. It is optimized for other things, not to waste energy and because of that it recognizes indeed that to play go very well but nothing else is rather less intelligent.

You do make a strong point there, specialized computers are stronger than humans in a specific task, but we are talking about general intelligence. I have to admit, too, that I have a hard time getting the bigger picture and get confused to easily. I didn't read any of the literature that would rather well define the problem, as the OP put it, so the discussion is likely less informative.

In my opinion, the comparison is unequal, still, because the Computer used a ton more resources and memory. There aren't enough go professionals to put together and let their averaged opinion learn and play, consuming as much energy.

Re: AI’s Language Problem

#225

Post by ctoth »

dmreedy wrote: post #134
Houshalter wrote: post #133

Earlier quoted context omitted.

>There is a class of functions that are simply not learnable without some prerequisite knowledge. This is directly analogous to a one-time pad in crypto. It is entirely possible that the function 'language' is in this class of unlearnable functions. I don't understand how this could possibly be the case. We can already make great progress on language understanding with simple methods like word2vec, or perhaps even ma…

It can be the case if Chomsky was right, and Universal Grammar and other similar structures are a thing. That would mean that part of our ability to understand language comes from the particular structure of our brain (which everyone seems to by and large share). That would mean that some of our ability to understand language is genetic in nature, by whatever means genes direct the structure of brain development.

Something like https://news.ycombinator.com/item?id=10016052 possibly?

Re: AI’s Language Problem

#226

Post by dlukes »

erikpukinskis wrote: post #84

No one would ever imagine that locking a baby in a featureless room with a giant stack of books would give them general intelligence. I don't understand why AI researchers think it will work for AIs. They need bodies that are biologically connected with the rest of the biosphere, with an intrinsic biological imperative, if they are ever to understand the world. I'm not saying they have to be exactly like us, but they…

In this vein, have you read about the work done by people mostly from the AI lab at the Vrije Universiteit in Brussels (Belgium)? (They're also affiliated with the Sony CSL in Paris: http://csl.sony.fr/language.php) They're precisely interested in the philosophical problem of how a grounded language emerges and is perpetuated among a population of embodied agents, as opposed to the engineering problem of, say, understanding complex, context-dependent natural language queries.

There's a great book which gives an overview of this field, The Talking Heads Experiment: Origins of Words and Meanings by Luc Steels, which discusses many of the advances made in this field (including for instance how having a grammar, as opposed to just stringing words related to what you want to say at random, is an evolutionary advantage because it boosts communicative success). It's published as open access, so go grab your free copy! :)

http://langsci-press.org/catalog/book/49

Chapter 4 in particular has a very interesting discussion of what's problematic with the machine learning approach -- that it takes a lot of training examples for a classifier to start making interesting decisions -- and presents a selectionist alternative to that, where distinctions (as in e.g. nodes in decision trees) are grown randomly and they're reinforced / pruned based on feedback. Crucially, the categories (semantic distinctions) are not labels given at the outset, but they emerge along with the language, based on the environment the agents encounter and the tasks they're using language for.

In general, I'd recommend Chapters 1 and 2 for a quick introduction, but in a pinch, I attempted to give a 50,000-foot summary in an essay I wrote (look under the heading Evolutionary Linguistics):

http://dlukes.github.io/cathedral-and-bazaar.html

I realize that engineering applications of these ideas might be a long way off (and perhaps they'll never materialize), but boy are these exciting discoveries about the very fabric of language :)

Re: AI’s Language Problem

#227

Post by mcphage »

Houshalter wrote: post #132
mcphage wrote: post #131

Earlier quoted context omitted.

> What is the meaning of a word, if not the relationship it has to other words? There's also the relationship it has with the world.

Well in my example the AI doesn't have to interact with the world at all. To pass the Turing test simply requires imitating a human, predicting what words they would say. You only need to know the relationships between words.

There are other tests for AI besides the Turing Test, some of which require more understanding on the part of the program. Check out Winograd Schemas: http://www.cs.nyu.edu/faculty/davise/papers/WinogradSchemas/... which hinge on understanding the subtleties of how words refer to the world.

Re: AI’s Language Problem

#228

Post by hacker42 »

brianchu wrote: post #218
hacker42 wrote: post #78

Earlier quoted context omitted.

There is evidence that language is fairly smooth though. For example, we can extract e.g. the gender vector from a word embedding space that is learned by a recurrent neural network. That seems to hint at the possibility that words, sentences and concepts live in smooth, high-dimensional manifold that makes them learnable for us in the first place (because in that case they can be learned by small local improvements…

The very method of using a word embedding space assumes the manifold is smooth, so the fact that vectors extracted from a method that assumes a smooth manifold, are in fact on a smooth manifold, is just circular and not evidence of anything.

The evidence is that this works in the first place.

Re: AI’s Language Problem

#229

Post by DonaldFisk »

Houshalter wrote: post #200
DonaldFisk wrote: post #180

Earlier quoted context omitted.

Wouldn't you still need to attach meanings to the words though? How could an AI system ever understand, for example, the Voynich Manuscript? There's plenty of text in it, and encryption methods when it was written weren't particularly strong. Or how would a person do if they were locked in a room with lots of books written in a language unknown to them?

Infact someone has run word2vec on the Voynich manuscript: http://blog.christianperone.com/2016/01/voynich-manuscript-w... (web archive while it's down: http://web.archive.org/web/20160205003817/http://blog.christ... ) Such methods could someday completely decode the thing, but for now they just show the relationships between words and different clusters of words, not their meaning. Of course we have no idea how the…

I'm familiar with this particular attack, as it was discussed here previously. It's a worthwhile attempt but the identification as star names, if real, hasn't been confirmed. But your reservations are justified.

More generally, has any attempt been made to identify the meanings of words in any sufficiently large corpus of text in a known foreign language (for example, Finnish), without being provided with a translation into English, and then compare the identified meanings to the actual meanings, as a first step towards translation?

Re: AI’s Language Problem

#230

Post by bweitzman »

dmreedy wrote: post #157
bweitzman wrote: post #149

Earlier quoted context omitted.

So if language comes from the structure of the brain, what would stop us from simulating that structure to give a machine mastery of language? And specifically what would imply that a machine which had some of that structure would need to learn by interaction as the top level comment suggests?

Nothing would stop us from simulating human brain-like (or analogously powerful) structures to build a machine that genuinely understands natural language. I'm arguing that we can't just learn those structures by statistical optimization techniques though. If it turns out that the easiest, or even only means of doing this is by emulating the human brain, then it is entirely possible that we inherit a whole new set of…

If there is an innate language structure in the brain then we know that it's possible to develop such a structure by statistical optimization, since this is exactly what evolution did, no?
Post reply on HN