Noam Chomsky on Where Artificial Intelligence Went Wrong • Hacker News Board

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#71

Post by andreasvc » Fri, Nov 02, 2012, 2:01 AM UTC

zmj wrote: post #17

thwest wrote: post #12

Earlier quoted context omitted.

How do you approach Heidegger's or Derrida's arguments that 'language is complicated, something magical must be going on'? Will you allow for no qualitatively different meaning to language than probabilistic associations of sounds with objects?

What could 'meaning' possibly be other than probabilistic associations of sounds and concepts?

The idea that meaning consists of associations is extremely primitive. It works for concrete nouns and verbs but it quickly fails as things get more complex. Language is used to refer to refer to abstract things, imaginary things, counterfactual situations, etc. And even if you do arrive at a series of concepts using associations, you have to understand how they are supposed to combine, even for completely novel sentences. In all these cases, there's arguably nothing there to associate with. I can't answer your question (I think no one can), but we can conclude that meaning is more than associations.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#72

Post by rdtsc » Fri, Nov 02, 2012, 2:21 AM UTC

AndyNemmity wrote: post #20

lsb wrote: post #6

The "right" way is to take endless numbers of videotapes of what's happening outside the video, and feed them into the biggest and fastest computer, gigabytes of data, and do complex statistical analysis -- you know, Bayesian this and that -- and you'll get some kind of prediction about what's gonna happen outside the window next. In fact, you get a much better prediction than the physics department will ever give. W…

Chomsky is agreeing that it's making a map. He just doesn't think that map is very useful on a scientific level, but is useful on an engineering level. You're responding that he's wrong, because it's useful on an engineering level. Right? I'm reading many comments here and they seem to keep boiling down to this notion. Am I wrong?

I think by default you'll find a disproportionate number of critics of Chomsky here. Some who understand what this about are more like to be engineers and like the engineering approach. Others who don't, saw Norvig's name and by default jumped to that side of the argument.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#73

Post by andreasvc » Fri, Nov 02, 2012, 2:22 AM UTC

rdtsc wrote: post #58

iandanforth wrote: post #10

Chomsky is right up there with Minsky in being part of the problem. His ideas about language being part of the genome are fanciful nonsense. Skinner produce reams of reproducible empirical observations of behavior which are, today, critical to the evaluation of the performance of AIs. Chomsky has produced interesting theories, but mostly derailed linguistics on the basis of the argument 'language is complicated, some…

> His contributions to science are negligible if not entirely detrimental. I think it would be nice to have some more references or more explanation before declaring Chomsky detrimental to science.

He dominated the field of linguistics for decades with anti-empiricist theories. You could argue that the popularity of his ideas held back alternative approaches; hence his contributions could be considered detrimental.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#74

Post by rdtsc » Fri, Nov 02, 2012, 2:57 AM UTC

MichaelGG wrote: post #57

reso wrote: post #53

Earlier quoted context omitted.

Chomsky revolutionized linguistics in the 1960s. Compiler theory incorporates a huge amount of his work. Calling his contributions negligible or detrimental is like saying Freud's contributions were the same.

Could you elaborate on Freud? I thought the general consensus was that none of his "theories" were accurate and evidence doesn't support or it directly contradicts his ideas. From the admittedly little I've read, it comes off as just "making stuff up" and generating frameworks that can explain anything (hence nothing). Psychoanalysis certainly has no place in modern psychology. Or are you saying that Freud, despite b…

I am not the grandparent author, but I think your last paragraph describes it well.

Yes looking back Freud's theories are bogus. However, that is only looking back and taking in consideration what has happened after. However, whatever happened after in the theory of how mind and human psychology was very much influenced by Freud. A lot of it was a reaction but it was still a reaction to something. Those that came after studied Freud and it was their background.

To say it another way, it hard to predict what would have happened if Freud was not there. We could have been in a worse shape maybe today.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#75

Post by lnanek2 » Fri, Nov 02, 2012, 3:01 AM UTC

zmj wrote: post #17

thwest wrote: post #12

Earlier quoted context omitted.

How do you approach Heidegger's or Derrida's arguments that 'language is complicated, something magical must be going on'? Will you allow for no qualitatively different meaning to language than probabilistic associations of sounds with objects?

What could 'meaning' possibly be other than probabilistic associations of sounds and concepts?

Chomsky claims language is built-in. So probabilistic associations are the exact opposite of his claims. Interestingly, there have been some very good baby studies that show babies inherently know statistics needed to learn probabilistic associations. You can show them a couple different color balls going into a box, then take out a bunch of the rare color that went in, and they will register more surprise even long before they can even talk. Things like that. Chomsky's assertions are, well, what you would expect from someone that old. They didn't understand neurons and biology and genetics so well then, so yay, magic things are possible!

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#76

Post by rdtsc » Fri, Nov 02, 2012, 3:06 AM UTC

andreasvc wrote: post #73

rdtsc wrote: post #58

Earlier quoted context omitted.

> His contributions to science are negligible if not entirely detrimental. I think it would be nice to have some more references or more explanation before declaring Chomsky detrimental to science.

He dominated the field of linguistics for decades with anti-empiricist theories. You could argue that the popularity of his ideas held back alternative approaches; hence his contributions could be considered detrimental.

> He dominated the field of linguistics for decades with anti-empiricist theories.

But he dominated. Where was Norvig or IBM's Watson back then?

Unfortunately things like these don't happen in a vacuum. You can only say looking back that this is didn't or that was a bad idea. You should have said that at that time when the theory was advanced or came up with a better one.

We didn't exactly live in in a totalitarian regime where say some Politburo dictated what the official theory about Genetics should be and everyone else gets sacked, and now finally we have freedom from oppression and we can get back on track.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#77

Post by jvm » Fri, Nov 02, 2012, 3:14 AM UTC

andreasvc wrote: post #68

jvm wrote: post #60

Earlier quoted context omitted.

CogSci PhD student here, I'm no fan of Chomsky but I'd still say his contributions were a lot more useful than Freud's. Freudian Psychology is mostly abandoned by practitioners at this point; the Chomsky hierarchy is still canon in CS.

The Chomsky hierarchy while being but a small part of his work is undoubtedly a useful technical contribution, but I don't see it as particularly relevant for cognition and linguistics. As an example, for cognition memory constraints seem to be much more fundamental than the type of formal rules. It is also rather arbitrary: many kinds of formal languages don't fit into his hierarchy, there are many other ways to car…

Sadly this all sounds quite correct.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#78

Post by rdtsc » Fri, Nov 02, 2012, 3:14 AM UTC

zmj wrote: post #17

thwest wrote: post #12

Earlier quoted context omitted.

How do you approach Heidegger's or Derrida's arguments that 'language is complicated, something magical must be going on'? Will you allow for no qualitatively different meaning to language than probabilistic associations of sounds with objects?

What could 'meaning' possibly be other than probabilistic associations of sounds and concepts?

Saying everything is just probabilistic associations is like saying everything is made from quanta of energy and thus every higher level concept or model is useless -- just simulate the quarks and you are set. Not only that -- simulate it by recording and observing the energy patterns going in and coming out of a black box.

Yes you can get some things to work and some to work well but the idea is that perhaps there is a better model that describes the mechanism or the encoding of meaning. That's what Chomsky is trying to say in this particular article. Some stopping at a brute force approach is a fine engineering approach but that doesn't mean everyone should, it is still worth trying to find a better model for it, if at least, just to gain an understanding.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#79

Post by rdtsc » Fri, Nov 02, 2012, 3:21 AM UTC

lnanek2 wrote: post #75

zmj wrote: post #17

Earlier quoted context omitted.

What could 'meaning' possibly be other than probabilistic associations of sounds and concepts?

Chomsky claims language is built-in. So probabilistic associations are the exact opposite of his claims. Interestingly, there have been some very good baby studies that show babies inherently know statistics needed to learn probabilistic associations. You can show them a couple different color balls going into a box, then take out a bunch of the rare color that went in, and they will register more surprise even long…

> Chomsky claims language is built-in.

In general he does. In this article he doesn't talk about he talks about approaches to AI.

> there have been some very good baby studies that show babies inherently know statistics needed to learn probabilistic associations.

Very good. Can we identify how that works and then build a robot that has the same mechanism in a more efficient way than simply simulating a brain at a molecular level. That is his argument here.

> They didn't understand neurons and biology and genetics so well then, so yay, magic things are possible!

So where were you 4-5 decades ago when he proposed his theory to propose a better one?

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#80

Post by rdtsc » Fri, Nov 02, 2012, 3:31 AM UTC

andreasvc wrote: post #71

zmj wrote: post #17

Earlier quoted context omitted.

What could 'meaning' possibly be other than probabilistic associations of sounds and concepts?

The idea that meaning consists of associations is extremely primitive. It works for concrete nouns and verbs but it quickly fails as things get more complex. Language is used to refer to refer to abstract things, imaginary things, counterfactual situations, etc. And even if you do arrive at a series of concepts using associations, you have to understand how they are supposed to combine, even for completely novel sent…

Isn't that Chomsky's argument here (in this article, not in his approach to linguistics in general)? -- That it is a good idea to try to find a better understanding of how the internal mechanisms work so we can build or simulate it better. You do that with carefully constructed experiments not with just observing inputs and outputs and training a neural network or a Markov model with it.