Norvig vs. Chomsky and the Fight for the Future of AI • Hacker News Board

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#21

Post by sp332 » Wed, Jul 25, 2012, 2:14 PM UTC

debacle wrote: post #13

sp332 wrote: post #9

Earlier quoted context omitted.

Children learn language much faster than it seems possible for a "blank" neural network to learn. It seems that there is some "circuitry" hard-wired into the human brain that helps learning language. So the question is: can a computer learn language as well as a human, without simply hard-coding language into it?

Children are definitely not a 'blank' neural network. They spend ~6 months staring off into nowhere, looking, listening, and slowly developing the skills to respond to their environment.

By "blank" of course I mean that they begin blank and immediately start learning from their environment. You say they develop skills "slowly" but it's still much faster than you would expect, unless children have some innate skill at language built in instead of being "blank."

Edit: sorry if this is vague, this is what I'm talking about https://en.wikipedia.org/wiki/Psychological_nativism

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#22

Post by bluekeybox » Wed, Jul 25, 2012, 2:16 PM UTC

orbitingpluto wrote: post #17

I know a card counter. I showed him how to condition probabilities to determine how to best play. He went for the full Monte Carlo method and he lets his simulation run for a week before he starts using it "just to make sure". It's frustrating because he doesn't get that his results are statistically significant after about 30 seconds of runtime. He still makes money doing it. The results are tangible, but he's still…

> You're not really a scientist if you create something that works and you don't really know why.

According to your logic, the only true "science" is mathematics. If you test the workings of your "creation" using scientific method, you're still a scientist. Scientific method is also about testing your claims empirically, and it has been successfully applied for more than a century to study of biological organisms, climate, and other complex systems that we do not "really" understand. Not to berate understanding of underlying mechanism which is always preferable, just to point out that there is more than one way to skin the cat.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#23

Post by knowtheory » Wed, Jul 25, 2012, 2:22 PM UTC

sp332 wrote: post #21

debacle wrote: post #13

Earlier quoted context omitted.

Children are definitely not a 'blank' neural network. They spend ~6 months staring off into nowhere, looking, listening, and slowly developing the skills to respond to their environment.

By "blank" of course I mean that they begin blank and immediately start learning from their environment. You say they develop skills "slowly" but it's still much faster than you would expect, unless children have some innate skill at language built in instead of being "blank." Edit: sorry if this is vague, this is what I'm talking about https://en.wikipedia.org/wiki/Psychological_nativism

You are being way too vague. What is setting your expectations of "slowly"? What rate would you expect children to learn language at? Even if that were to be the case, your argument is essentially a god of the gaps argument. Not P therefor Q is not sound reasoning.

The whole notion of the Universal Grammar and innate language faculties which instantiate subsets of the Universal Grammar is weird.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#24

Post by robg » Wed, Jul 25, 2012, 2:25 PM UTC

This is one of those rare moments in intellectual life where being in the room and now seeing the debate develop, it becomes clear that the resulting hype isn't (wasn't) loud enough.

This distinction marks the real turning point in AI from abstract, grand claims with highly restrictive evidence toward engineering that simply works. Who cares about the ontology when we can recreate? It's like saying airplanes don't properly explain flight because they don't replicate how birds do it. Who cares? We can fly (and translate and soon reason) artificially.

It's clear that Chomsky and Universal Syntax has held back the entire field of AI (and at MIT). There isn't one algorithm in the human mind to decode all of our mental capabilities. That's mistaking subjectivity for objective lessons. Trying to recreate that Phantom has led to rule tables in AI, constraints on how the mind must operate. Instead, by allowing those fuzzy boundaries to accumulate with evidence, statistical approaches win in the long-term of our lives and in this debate.

Kuhn knew what happens to dinosaurs.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#25

Post by netcan » Wed, Jul 25, 2012, 2:32 PM UTC

debacle wrote: post #2

They have two different definitions of "artificial intelligence," which is where the schism seems to be arising from. Chomsky takes the academic approach - artificial intelligence is the simulation of humanlike (or even possibly mammalian) intelligence. Norvig is taking the engineering approach - artificial intelligence needs only to pass the Turing test. They're both right, both approaches have value, and they both…

I'm going out on a limb because I don't really know much about neurology & I may be wrong about facts but.. I think an issue here is that we don't really know what "natural intelligence" is.

For a significant part of the Scientific age we knew about genes in some sense without knowing much about them. We called them traits, observed & measured them. We got to know some "rules" about their inheritance. But it wasn't until genetics got to be a little better understood that we got to know their physical manifestation. We can explain the diference between genetic & cultural (memetic?) inheritance in these terms. A descendant's cooking habits are memetic and her hair colour is genetic.

When it comes to neuroscience I think we're where we were a century ago in biology. Emotions, thoughts, memories. We don't know what their physical manifestation is. We dont know how they work. Since we don't know much about how natural intelligence works I think our common sense definition of intelligence is, to a certain extent: "stuff we can do that computers can't."

I think that if we had a definition that was more functional than observational, you wouldn't be hesitant at all to use "mammalian" in your definition. Whatever processes result in observed human intelligence are almost certainly shared with other species. We'd probably also know what species to draw the lines at: reptiles? invertebrates? fungi?

If apes have intelligence, goldfish don't but octopi do that suggests there multiple versions of natural intelligence.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#26

Post by _delirium » Wed, Jul 25, 2012, 2:37 PM UTC

robg wrote: post #24

This is one of those rare moments in intellectual life where being in the room and now seeing the debate develop, it becomes clear that the resulting hype isn't (wasn't) loud enough. This distinction marks the real turning point in AI from abstract, grand claims with highly restrictive evidence toward engineering that simply works. Who cares about the ontology when we can recreate? It's like saying airplanes don't pr…

I don't think I would take that strident battle-against-dinosaurs view, in part because I think intellectually understanding things is useful in and of itself, not some kind of "just build it and shut up" anti-intellectual view; and also because I don't think it's an accurate summary of the history of AI. There's no particular reason we can't both build and study things in various ways, and the history of AI has been full of people doing many takes on both.

In particular, statistical approaches have been used for a long time, but were not practical until fairly recently; it was the lack of "big data" computing power holding them back more than anything. Statistical machine translation and parsing experiments have been tried on and off for decades, but with 1950s-era data they produced total garbage as output, even worse than the (also bad) symbolic approaches. Hence why Shannon's work on text processing didn't produce practical NLP or NLG systems. It took Google-sized data to produce statistical translation that was actually usable.

What numerical approaches were possible on computers of the time were fairly extensively investigated when they became possible (e.g. the 1980s focus on "sub-symbolic AI", with perceptrons, neural networks, numerical regression methods, etc.). Some were shelved for years because they just didn't work as well, e.g. symbolic game-tree search massively outperformed machine learning in board games in the early experiments, which is why Samuels's 1950s ML-based checkers player was theoretically intriguing but not considered very practical.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#27

Post by knowtheory » Wed, Jul 25, 2012, 2:39 PM UTC

It's a little bit frustrating to read a rehash of an argument that was cutting edge maybe back in the late 90s, especially one that is so poorly written, and framed as a battle between two intellectuals.

Chomsky's past his heyday. He has been seminal in his field, but he's no longer doing research which pushes at the boundaries of our understanding of language, how to model it, or what the fundamental nature of language understanding systems is. (as one might infer, I come from a non-chomskyian school of linguistics).

Given that we have actual data and research about large scale systems that do interesting things (including the massive artificial neural network that google built last month, see: http://www.wired.com/wiredscience/2012/06/google-x-neural-ne... ) reporting as substance free and obfuscating as this is, is a real frustration, when we could be talking about more interesting things, such as what a solid operational definition of meaning is, or how exactly heuristic/rule based systems actually differ from statistical mechanism, and whether or not all heuristic systems can (or should) be modeled with statistical systems.

The framing of this article is particularly galling because there are so many non-chomskian linguists out in the world who operate fruitfully in the statistical domain. Propping Chomsky up as somehow representative of all linguists is pretty specious and a bit irritating.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#28

Post by cma » Wed, Jul 25, 2012, 2:40 PM UTC

debacle wrote: post #13

sp332 wrote: post #9

Earlier quoted context omitted.

Children learn language much faster than it seems possible for a "blank" neural network to learn. It seems that there is some "circuitry" hard-wired into the human brain that helps learning language. So the question is: can a computer learn language as well as a human, without simply hard-coding language into it?

Children are definitely not a 'blank' neural network. They spend ~6 months staring off into nowhere, looking, listening, and slowly developing the skills to respond to their environment.

A newborn baby will recoil away from a dark circle growing larger on a screen. Where did he learn to associate a growing share with an object approaching near enough to collide?

We have certain built in behaviors and reflexes.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#29

Post by radarsat1 » Wed, Jul 25, 2012, 2:41 PM UTC

sp332 wrote: post #9

phuff wrote: post #3

Earlier quoted context omitted.

Having studied Chomsky a fair bit in grad school, and also studied cognitive linguistics a fair bit in grad school, I think the idea that Chomsky's models will ever win anything is just wrong. Chomsky's central problem is that his modeling is not based on anything biological at all. His models don't correspond to reality. Some of them were based on some assumptions about how the brain works that were untestable in th…

Children learn language much faster than it seems possible for a "blank" neural network to learn. It seems that there is some "circuitry" hard-wired into the human brain that helps learning language. So the question is: can a computer learn language as well as a human, without simply hard-coding language into it?

I'm not sure what you mean by "faster," (what are you comparing to exactly?), but I think something that speeds up human learning considerably as compared to computers is feedback. Children don't just blankly sit there taking in information and then "fitting it" to a model-- they perform actions and observe the consequences; it is empirical. The embodied action-perception loop is fundamental to how real-world learning works. A closer computer model is reinforcement learning, for example, which does exactly this, it wraps a neural network in an action-perception loop an uses online training to learn the reward function. The problem is of course that the reward function can be very hard to design except for fairly simple tasks.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#30

Post by brudgers » Wed, Jul 25, 2012, 2:41 PM UTC

Intellectually, there seems to be something as wrong with avoiding anthropomorphism when discussing human endeavors (such as language) as there is with anthropomorphic explanations of erosion or chemical reactions. Skinnarian approaches to language may leave people unsatisfied because there is no story, just clinical observation.

Norvig's approach (as characterized in the article) takes the the "Artificial" in "Artificial Intelligence" to include the mechanism by which an intelligence makes decisions. Chompsky's aesthetic of linguistics applied to AI would treat "Artificial" as a description of the platform in which an intelligence is embodied (i.e. non-biological) while requiring the platform to operate linguistically on the same principles as a "natural intelligence."

Norvig's approach (as characterized in the article) is essentially a better Eliza (or Ford's faster horse).

If one takes the Turing Test as scientifically meaningful rather than an engineering standard, then one falls in one camp or the other and the Norvig Chompsky debate is over a pseudo-problem. "Artificial Intelligence" is in that sense metaphysical jargon.