Chomsky's argument here is basically the same argument he used to reinvent linguistics in the 50's and 60's, so understanding what's
really being said here depends on a bit of context. Since neither previous discussions nor this article has quite nailed it, the discussion is indeed worth having (sorry, ColinWright).
The crux of the debate is this. When Chomsky was a grad student at UPenn most linguists thought that language was learned by a complicated mimicry -- that we learn language by imitating behavior, similar to how birds learn to call. The "hard problems" of linguistics were completely solved and linguistics had become a primarily classificatory science, with linguists simply cataloging words into parts of speech. This line of thought was interchangeably known as behavioralism or empiricism.
One of Chomsky's transformative insights was that most sentences that are reasonably long are completely unique in human history, and will also never again be uttered by another person, ever. (For example, try Googling that exact sentence.) One consequence of this is that people realized that the mimicry argument could not really account for the robust well-formed structure of sentences and give us an infinite set of them. What we need to generate an infinite set of well-structured sentences is a grammar. Thus the universal grammar was born, and while Chomsky did not convince all the prominent linguists of the time, he did convince all their grad students, and the field of linguistics has seldom looked back.
Where this begins to intersect with AI is where Chomsky is usually criticized for not having been quite revolutionary enough. His outline of linguistics basically split the field into semantics (which studies the meaning of language) and syntax (which studies the structure of language). He argued that everything about language that must be interpreted (like meaning) must go on the semantics side of the line, and everything else should go on the other side of the line. He does not believe syntax to be interpretive at all, and tends to react violently when anyone tries to push empiricism into the syntax dialogue. Even a lot of his students don't buy that syntax is completely not-interpretive, and so someone like Lakoff would claim that if he was revolutionary, he was not quite revolutionary enough.
Here's what this means for AI. Chomsky sees the statistical approach to learning as a type of empiricism. You take a corpus, learn some stuff statistically, and then perform well on a task. To someone like Chomsky this probably looks like Skinner's old model, but instead of words like "mimicry", we use words like "statistical inference." Remember that empiricism and syntax should be strictly separate, and it becomes easy to see why something like this would make him cranky.
Of course, computer scientists like Norvig and linguists like Lakoff disagree. Their collective argument is that some aspects of syntax are indeed interpretive, and that (in the case of Norvig) they can be learned statistically (using, e.g., PCFGs). For example consider the sentence "John called Mary a Republican and then SHE insulted HIM". This really only makes sense if you presume that the participants think that "Republican" is an insult, but how do you know that? The answer seems to be through some sort of past experience, which Norvig would say can be and should be modeled statistically. And that in short is the debate and why it exists.
The other complaints with what Chomsky said here are that it's scientifically incorrect. Chomsky claimed, for example, that statistical models are basically not real science, and not used in the history of science, which is obviously wrong. Norvig pointed out, for example, that in physics sometimes our only choice is to infer something statistically, as in the case of the gravitational constant or the Higgs boson. But given Chomsky's history, it's fair to assume he meant this in the context of behavioral science, in which case his point is mostly true (modulo the "old" model of linguistics, which he hates). But of course it's important to remember that his model of linguistics was also without precedent in the history of science, so that alone is not really justification for his position.