Norvig vs. Chomsky and the Fight for the Future of AI • Hacker News Board

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#31

Post by _delirium » Wed, Jul 25, 2012, 2:43 PM UTC

debacle wrote: post #2

They have two different definitions of "artificial intelligence," which is where the schism seems to be arising from. Chomsky takes the academic approach - artificial intelligence is the simulation of humanlike (or even possibly mammalian) intelligence. Norvig is taking the engineering approach - artificial intelligence needs only to pass the Turing test. They're both right, both approaches have value, and they both…

A significant proportion of the "academic approach" is actually a third one: artificial intelligence is the analysis and implementation of rational decision-making. That approach tends to care neither about biological accuracy, nor believability in a Turing-test sense. Rather, it cares about whether its decisions are correct based on evidence available to the decision-maker. That's the kind of attitude you most often find in both statistical and logic-based AI circles.

Actually, Russell & Norvig's AI textbook has a nice summary of these different approaches to AI in its intro chapter.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#32

Post by waterlesscloud » Wed, Jul 25, 2012, 2:45 PM UTC

jan_g wrote: post #4

debacle wrote: post #2

They have two different definitions of "artificial intelligence," which is where the schism seems to be arising from. Chomsky takes the academic approach - artificial intelligence is the simulation of humanlike (or even possibly mammalian) intelligence. Norvig is taking the engineering approach - artificial intelligence needs only to pass the Turing test. They're both right, both approaches have value, and they both…

But what exactly is true artificial intelligence? For example, I consider Google search and Wolfram alpha very intelligent. They can do math, answer questions, rank information, follow current events, ...

I think statistical models are an 80% solution. They get a long way down the path very quickly, but then they hit a wall and don't advance much further. Search, translation, probably also autonomous vehicles. They get to a point with statistical approaches and then they stop advancing.

The rapid success at first may be leading to a dead end.

Having said that, there's a lot of value in these technologies as assistance to human intelligence, but I'm skeptical they're ever going to lead to full-on autonomous intelligence.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#33

Post by Jun8 » Wed, Jul 25, 2012, 2:48 PM UTC

sp332 wrote: post #9

phuff wrote: post #3

Earlier quoted context omitted.

Having studied Chomsky a fair bit in grad school, and also studied cognitive linguistics a fair bit in grad school, I think the idea that Chomsky's models will ever win anything is just wrong. Chomsky's central problem is that his modeling is not based on anything biological at all. His models don't correspond to reality. Some of them were based on some assumptions about how the brain works that were untestable in th…

Children learn language much faster than it seems possible for a "blank" neural network to learn. It seems that there is some "circuitry" hard-wired into the human brain that helps learning language. So the question is: can a computer learn language as well as a human, without simply hard-coding language into it?

This is a popular summary of Chomsky's thesis that was put down decades ago, when cognitive psychology was at its infancy. Now we know a lot more on how babies learn the world and language (do a Google search on "infant statistical learning") and most evidence points to the fact that they employ algorithms that are mainly statistical in nature for learning.

"Children learn language much faster than it seems possible for a "blank" neural network to learn." This is a very strong statement that has no mathematical or computational proof AFAIK. It had no proof when Chomksy first put down that thesis either, it was an axiom of his. BTW, belief in a specialized language faculty was not universal, even in the past, esp. some philosophers of language disagreed with this view.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#34

Post by knowtheory » Wed, Jul 25, 2012, 2:51 PM UTC

mootothemax wrote: post #8

Isn't this basically an argument over John Searle's Chinese Room thought experiment? It supposes that there is a program that gives a computer the ability to carry on an intelligent conversation in written Chinese. If the program is given to someone who speaks only English to execute the instructions of the program by hand, then in theory, the English speaker would also be able to carry on a conversation in written C…

Just as a note, the space of possible responses to Searle's argument have been pretty well enumerated here: http://plato.stanford.edu/entries/chinese-room/

I'm of the opinion that the room has an understanding entity inside of it, in talking about infinitely-sized books with an infinitely-sized index allowing any mechanical process to map an input to a correct output, you've hypothesized something complicated enough that it should be said to be an entity capable of understanding/meaning.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#35

Post by gnaritas » Wed, Jul 25, 2012, 2:53 PM UTC

knowtheory wrote: post #23

sp332 wrote: post #21

Earlier quoted context omitted.

By "blank" of course I mean that they begin blank and immediately start learning from their environment. You say they develop skills "slowly" but it's still much faster than you would expect, unless children have some innate skill at language built in instead of being "blank." Edit: sorry if this is vague, this is what I'm talking about https://en.wikipedia.org/wiki/Psychological_nativism

You are being way too vague. What is setting your expectations of "slowly"? What rate would you expect children to learn language at? Even if that were to be the case, your argument is essentially a god of the gaps argument. Not P therefor Q is not sound reasoning. The whole notion of the Universal Grammar and innate language faculties which instantiate subsets of the Universal Grammar is weird.

No, he's right, check out Pinkers the language instinct for the full treatment on the issue. Children learn at a rate impossible from just what they hear and imitate. We are pre-wired for language.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#36

Post by sp332 » Wed, Jul 25, 2012, 2:54 PM UTC

Jun8 wrote: post #33

sp332 wrote: post #9

Earlier quoted context omitted.

Children learn language much faster than it seems possible for a "blank" neural network to learn. It seems that there is some "circuitry" hard-wired into the human brain that helps learning language. So the question is: can a computer learn language as well as a human, without simply hard-coding language into it?

This is a popular summary of Chomsky's thesis that was put down decades ago, when cognitive psychology was at its infancy. Now we know a lot more on how babies learn the world and language (do a Google search on "infant statistical learning") and most evidence points to the fact that they employ algorithms that are mainly statistical in nature for learning. "Children learn language much faster than it seems possible…

It follows logically from these 3 premises https://en.wikipedia.org/wiki/Poverty_of_the_stimulus#Summar... Those premises could be wrong but it's not just an axiom.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#37

Post by bugsbunnyak » Wed, Jul 25, 2012, 2:56 PM UTC

debacle wrote: post #2

They have two different definitions of "artificial intelligence," which is where the schism seems to be arising from. Chomsky takes the academic approach - artificial intelligence is the simulation of humanlike (or even possibly mammalian) intelligence. Norvig is taking the engineering approach - artificial intelligence needs only to pass the Turing test. They're both right, both approaches have value, and they both…

> Chomsky's approach, on the other hand, will result in a 'true' artificial intelligence, the way neurologists understand it. It's just going to take a lot longer to get there.

High-level behavioral impressions taken by a neurologist are a convenient abstraction. That this high-level behavior is useful in monitoring mental state (outputs) says very little about the underlying 'hardware'. In fact, this is the `fundamental` debate in cognitive science: from whence does intelligence arise? Theories generally fall under two headings: 'top-down' and 'bottom-up', which roughly correspond to 'pre-programmed' and 'emergent'. The canonical bottom-up approach is the neural network, approximating cells with various equations that govern behavior (outputs) based on aggregate input (there are various levels at which this can be done). There are a variety of top-down approaches, a typical approach would take the form of logic engines (think Prolog), or generative rules (Chomsky)

Statistical modelling approaches are closer to bottom-up, but depending on the model they may still incorporate domain knowledge that is emergent from the model input.

Statistical approaches have momentum these days due to considerable success - thanks largely to Moore's law. However, they also have biological support: what is a neuron? It's an FPGA with a lot of electrical and chemical inputs. Small neural circuits can behave statistically, and it's an open question whether this gives rise to high-level behavior. A big reason it's an open question is that we don't yet have the spatial or temporal resolution to measure enough signals.

That said, there is plenty of room for what I consider a happy medium: locally statistical behavior, but globally (and generationally) top-down organization driven by genetics.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#38

Post by jan_g » Wed, Jul 25, 2012, 2:59 PM UTC

waterlesscloud wrote: post #32

jan_g wrote: post #4

Earlier quoted context omitted.

But what exactly is true artificial intelligence? For example, I consider Google search and Wolfram alpha very intelligent. They can do math, answer questions, rank information, follow current events, ...

I think statistical models are an 80% solution. They get a long way down the path very quickly, but then they hit a wall and don't advance much further. Search, translation, probably also autonomous vehicles. They get to a point with statistical approaches and then they stop advancing. The rapid success at first may be leading to a dead end. Having said that, there's a lot of value in these technologies as assistance…

But when does your average human hit a wall? Pretty quickly, I'd say. Mostly due to laziness and parenting and getting old.

I think that you are also getting at the _sentience_. I think this is what most people refer to when speaking about true AI. You know, having consciousness, desires, social skills, etc.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#39

Post by Retric » Wed, Jul 25, 2012, 3:07 PM UTC

sp332 wrote: post #36

Jun8 wrote: post #33

Earlier quoted context omitted.

This is a popular summary of Chomsky's thesis that was put down decades ago, when cognitive psychology was at its infancy. Now we know a lot more on how babies learn the world and language (do a Google search on "infant statistical learning") and most evidence points to the fact that they employ algorithms that are mainly statistical in nature for learning. "Children learn language much faster than it seems possible…

It follows logically from these 3 premises https://en.wikipedia.org/wiki/Poverty_of_the_stimulus#Summar... Those premises could be wrong but it's not just an axiom.

#2 is simply wrong. Children are corrected when their grammar is off so they do get to see incorrect sentence structure. It's questionable if children could learn language from only watching TV, but that's not the standard learning environment.

PS: I am far from the first person to point that out. At this point they are treating it as an axiom because they continue to believe it despite the disproof of their premises.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#40

Post by phaedrus » Wed, Jul 25, 2012, 3:10 PM UTC

I spent about ten years working on Markov based chat programs. I gave up on themwhen I realized that no matter how sophisticated your statistical model it will never be more than a statistical analysis of text, unless it includes some rich rule based model of mental processes and mental objects. It may be that such a model of mental processes must itself be fuzzy and probabilistic, but it must exist. Therefore I come down firmly on the side of Chomsky in this debate: we should pursue theories of intelligence, and stastical models without any theory do not advance our scientific understanding of AI, however practical their application may be at the present time. This is not to say statistical methods do not work, of course they work, what I am saying is it is not a path that leads to true understanding of intelligence any more than spectral analysis of the EMF emissions of a running computer would lead to a theory of computation.