Noam Chomsky on Where Artificial Intelligence Went Wrong

To some extent, I agree. But for some reason, I have a feeling that "Wearable Computing" is gonna have similar fate.

This reminds me a bit of what scientists in other fields refer to as "empirical equations", which are equations fit from data without any particular theoretical backing or reason to believe that their components are a good model of reality. They're useful in that they may predict observations well, especially over a specific range of observables, but they don't necessarily give us an understanding of what's going on. An example is the historical Prony equation for hydraulic pressure loss (http://en.wikipedia.org/wiki/Prony_equation), which doesn't actually correctly model what happens in fluid flow, but does happen to empirically produce fairly good results over a range of values, partly through the use of two magic numbers fit from data.

Another example might be the winning entry in the Netflix competition. If your goal is to predict film preferences (which is Netflix's goal, of course), it appears to give pretty good estimates. But I don't think even its authors would claim that it's a a scientific model of how humans form preferences.

In both cases the underlying problem is that there are fairly general functional forms, such as a few terms of a Taylor series, or a forest of decision trees, that can empirically model almost anything to a certain degree of accuracy, given enough data, even if the underlying process looks nothing like them. Therefore they can give accurate predictions that work in practice, without being accurate models of what's happening in the underlying system. Chomsky appears to be of the opinion that statistical NLP systems are more of that variety, so may be good engineering solutions without being good scientific models.

Exactly. Well said, and clearly articulates the point here.

What I don't get is, what Chomsky is saying is all together the standard, and yet people are insulting him for what is all together a very simple idea you expressed plainly.

Yes, NLP systems are going to have many engineering uses, and Chomsky agrees. Are they going to help in the true scientific understanding of the systems?

It's unlikely. It's likely to be "good engineering solutions without being good scientific models" as you elegantly put it.

Your observation is so vague and general to the point of being rather meaningless. Almost every physical theory is described by an underlying mapping between inputs.

The interesting point is the expression power of your model. : to take an example I am somehow familiar with, current large vocabulary speech recognizers have millions of parameters. They work relatively well, but they are very difficult to interpret, and it is hard to see how they help us understanding how speech recognition actually works in our brain.

To make a somehow flawed analogy, every Turing complete language is equivalent, but getting the machine code of a very large project is not very interesting if you want to understand it, while it is mostly enough if you just want to use it.

Noam Chomsky irritates me here's why - he's vague, so astonishingly vague that he can hide his uselessness within it.

> Chomsky derided researchers in machine learning who use purely statistical methods to produce behavior that mimics something in the world, but who don’t try to understand the meaning of that behavior. Chomsky compared such researchers to scientists who might study the dance made by a bee returning to the hive, and who could produce a statistically based simulation of such a dance without attempting to understand why the bee behaved that way.

-- http://www.tor.com/blogs/2011/06/norvig-vs-chomsky-and-the-f...

What does that even mean?

> But the number of parameters in his theory continued to multiply, never quite catching up to the number of exceptions, until it was no longer clear that Chomsky’s theories were elegant anymore. In fact, one could argue that the state of Chomskyan linguistics is like the state of astronomy circa Copernicus: it wasn’t that the geocentric model didn’t work, but the theory required so many additional orbits-within-orbits that people were finally willing to accept a different way of doing things. AI endeavored for a long time to work with elegant logical representations of language, and it just proved impossible to enumerate all the rules, or pretend that humans consistently followed them. Norvig points out that basically all successful language-related AI programs now use statistical reasoning

> But his fundamental stance, which he calls the “algorithmic modeling culture,” is to believe that “nature’s black box cannot necessarily be described by a simple model.” He likens Chomsky’s quest for a more beautiful model to Platonic mysticism, and he compares Chomsky to Bill O’Reilly in his lack of satisfaction with answers that work. “Tide goes in, tide goes out. Never a miscommunication. You can’t explain that,” O’Reilly once said, apparently unsatisfied with physics as an explanation for anything.

-- http://www.tor.com/blogs/2011/06/norvig-vs-chomsky-and-the-f...

AI went wrong when Chomsky came around with his rule based translation ideas that were hideously wrong and probably set us back 20 years - see here:

http://norvig.com/chomsky.html

He's a more irritating linguistic version of Richard Dawkins (who doesn't have an active research career).

> Every time I fire a linguist, the performance of the speech recognizer goes up

-- http://en.wikipedia.org/wiki/Frederick_Jelinek