Norvig vs. Chomsky and the Fight for the Future of AI

I am a entrepreneur/researcher working to create artificial intelligence. My approach follows Turing's suggestion that one should create a child mind and proceed to educate it. I employ Construction Grammar in my English dialog system - not a statistical parser/generator. Operating on a smartphone, I use available statistical speech recognition engines to transform speech to text, but from that point onwards the server-side processing in Construction Grammar is symbolic, thus engineered from first principles. Likewise, for English generation, my discourse planner emits structured RDF that the bi-directional Construction Grammar generator transforms into a text utterance. That symbolic text is then input to an available statistical text-to-speech engine available on the smartphone, to speak to the user.

As an example of the power of symbolic approaches, my parser has a complete symbolic analysis of English auxillary verb constructions, producing unique, meaning-rich, RDF-compatible semantics for:

I am learning about computers.

We are learning about computers.

We will be learning about computers.

I could be learning about computers.

I have been learning about computers.

I better learn about computers.

I had better learn about computers.

I dare learn about computers.

I did learn about computers.

I do learn about computers.

He does learn about computers.

I had learned about computers.

He has learned about computers.

I have learned about computers.

He is learning about computers.

I need learning about computers.

I ought to learn about computers.

I ought to be learning about computers.

I used to learn about computers.

I was learning about computers.

We were learning about computers.

Because of the so-far limited success of my work, I am inclined to agree with Chomsky's AI argument despite using a modern grammar opposed to his linguistic principles.

An artificial intelligence will use both statistical techniques and symbolic, e.g. procedural techniques, I think. With the most useful intelligent behavior being symbolic. E.g. an AI designing, writing and testing software.

Intellectually understanding things is useful. There's no anti-intellectualism here.

It's become increasingly clear that Chomsky's approach to language is not going to generalize to AI successes or explanations of many domains of human performance, as was promised. That approach has not yielded fruitfully anywhere except the narrow realm of syntax (in particular typologies of syntax). While machine learning has kicked butt left and right.

There is no reason people can't go on studying syntax at the same time machine learning expands. But it would be dishonest to ignore the extreme and exclusive claims laid down by Chomsky's school at the outset of the cognitive revolution. These claims were given a very liberal benefit of the doubt for decades. A lot of good work has been done in syntax. But now those claims about AI and psychology-in-general are clearly threadbare, since they have not yielded the promised fruits either in AI or in explaining human functions. They have not even yielded plausible IOUs. Remember, this was supposed to explain pretty much everything. Not even language acquisition has been explained.

That is empirical inadequacy. Who cares if Chomsky thinks it's beautiful?

And it's equally clear that learning of various kinds has to occur and also gives us the more parsimonious and elegant solutions to problems (no-solution and no-explanation is not elegant even if it is simpler).

Machine learning has a great deal of conceptual beauty - if you study it rather than pooh-poohing it because of some facile abductive argument to UG.

Computing power isn't a problem. The scale of the human brain is enormous and we are still nowhere near reaching it with our computational resources.

As far as language modeling, this is a recent paper that models language on the character level rather than word level and can track long term dependencies and even generate plausible sounding non-words from time to time: http://www.cs.toronto.edu/~ilya/pubs/2011/LANG-RNN.pdf

The state of the art is improving a bit, although this method still knows nothing of meaning so it can often generate some strange sentences. Still, I wouldn't write off the whole field yet--just because something didnt work with tools years ago doesn't mean it isn't possible.

You are right, these statistical approaches are algorithms. And they themselves are very simple and do not necessarily require a lot of initial state. And you are also right that it is extra hard to reverse-engineer these.

But they amount to learning and search. And because of the politics of Chomsky's movement, it has always fought the concept of learning tooth and nail in favor of innate knowledge or ready-made models. This is why we talk about language "acquisition" and why we read Chomsky saying that contemporaries who studied learning were facilitating totalitarianism.

It's also worth mentioning that Chomsky has stated that he does not think the brain has evolved.

So however many algorithms there are (and however they are employed to get the vast array of intelligent human actions) many things you have said here are totally anathema to Chomsky's school.

It was pushed too hard, it overextended and only now has any challenge come to it - from the quarter of AI. (Up until a few years ago, fighting it in academic Psychology meant you were OK with killing your career, and anyone less respectable than Norvig would get torn apart - look at what they are saying about Norvig now).

There is no valid argument leading FROM your disenchantment with Markov based chat programs, TO a conclusion that machine learning is invalid. Markov chatters are a toy.

Did you produce a better chat bot based on UG? - No? Then on what basis are you junking machine learning?

Machine translation is far from a solved problem. But Chomsky's school claimed they were going to solve it in the 60s or perhaps the 70s. Do you know what is the basis for the most successful current approach to machine translation?

Analysis of text. (But not the kind of simplistic junk one does in a Markov chatter)

All you have done is suggest that some beautiful perfect text model exists natively in every person. (Presumably this evolved somehow - or if you are Chomsky, it just developed like a crystal for no apparent reason). This isn't an explanation of anything unless you actually find that model instantiated in the brain. But this is just not happening. So either our instruments are still too crude to detect it, or it's not really there.

Appealing to an as-yet unknown perfect universal text model does not build a better chat bot or a better explanation of human behavior.

True understanding of intelligence must incorporate an understanding of how learning occurs. Because anyone who watches children sees learning occurring, and only doctrinaire Chomskyists deny that it occurs (because it is not beautiful enough and some abductive argument is claimed to show that it is not sufficient).