Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Norvig vs. Chomsky and the Fight for the Future of AI

tor.com

Post reply on HN
Print view View original

61–70 of 152 posts

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#61

Post by JabavuAdams »

phaedrus wrote: post #40

I spent about ten years working on Markov based chat programs. I gave up on themwhen I realized that no matter how sophisticated your statistical model it will never be more than a statistical analysis of text, unless it includes some rich rule based model of mental processes and mental objects. It may be that such a model of mental processes must itself be fuzzy and probabilistic, but it must exist. Therefore I come…

Two experiences have made it clear to me that humans don't understand language that well, without context:

(1) Raising a child. My dad often remarks that he's surprised that my daughter knows how to use a word in just the right context. I'm not, because this is a natural product of mimicry: if you copy what others say, you usually use the words in the correct context. As with computer-generated text the exceptions are often hilarious.

(2) Song lyrics. I had a very clear experience where I just could not understand the refrain of Gold Guns Girls. It sounded completely unintelligible until I read the lyrics. After that, it sounded crystal clear. Why would reading the lyrics make the song sound different? Context.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#62

Post by VikingCoder »

I picture Chomsky as Kepler, trying to build orbits out of Platonic solids.

Until Kepler had access to Brahe's data, he was not going to be able to come up with his theories of planetary motions.

Worse than that, the laws of planetary motion present a simplistic view of the universe: what happens when a bunch of small objects orbit a very massive object. I think they wouldn't help you out at all, in trying to understand planets moving in a binary star system.

There is no analytic solution to the N-body problem. We can only simulate the motions of a group of massive bodies by iteratively applying the laws of gravitation that we have deduced. Knowing the mathematical properties of how objects behave in a gravitational field, and actually understanding HOW GRAVITY WORKS are two enormously different things. Newton was frustrated with the theory of Gravity, because it was, as Norvig's models, just a model - with no explanation of why. But the model allows you to make falsifiable predictions, and understand how the universe will behave. Looking for the Higgs Boson is awesome - but there is potentially no equivalent in the linguistic world.

Chomsky asks us to ignore F = G * m1 * m2 / r^2, because there's no WHY attached to it.

PS - this understanding of the history of science is brought to you by Carl Sagan's Cosmos TV series. I have no deeper insight than that.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#63

Post by JabavuAdams »

robg wrote: post #24

This is one of those rare moments in intellectual life where being in the room and now seeing the debate develop, it becomes clear that the resulting hype isn't (wasn't) loud enough. This distinction marks the real turning point in AI from abstract, grand claims with highly restrictive evidence toward engineering that simply works. Who cares about the ontology when we can recreate? It's like saying airplanes don't pr…

> There isn't one algorithm in the human mind to decode all of our mental capabilities.

Careful here. There could be one algorithm, but we may not be able to express its parameters in the way we like to for simple models.

These statistical approaches are algorithms. It's the fact that we can't make sense of the parameters that seems to lead people to believe that they don't explain anything.

The fact that we can't categorize all life in an unambiguous way that makes sense to humans doesn't mean that the elegant and simple algorithm of evolution by natural selection is wrong.

Model vs. initial conditions. Knowledge vs. search.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#64

Post by OmegaHN »

VikingCoder wrote: post #62

I picture Chomsky as Kepler, trying to build orbits out of Platonic solids. Until Kepler had access to Brahe's data, he was not going to be able to come up with his theories of planetary motions. Worse than that, the laws of planetary motion present a simplistic view of the universe: what happens when a bunch of small objects orbit a very massive object. I think they wouldn't help you out at all, in trying to underst…

I think it is the other way around. Chomsky is trying to find the underlying structure of intelligence (just like gravity underlies planetary motion), and is saying that others are simply trying to generate a model of intelligence (through statistical methods) with no understanding of why the intelligence behaves that way. Gravity is the why, planetary motion is the model produced by data (acquired by Brahe).

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#65

Post by why-el »

Jun8 wrote: post #43

OK, let me start with two facts, one objective, one personal: (i) Noam Chomsky is a genius with many contributions to linguistics and computer science (ii) I think his overall influence had been damaging to linguistics. Here's a summary of Chomsky's career in layman's terms: As everyone knows, Chomsky first came to prominence with his critique of Skinner (who, as everyone also knows, was a total psycho). He pretty mu…

You cant label something as a fact then proceed to say "I think". (ii) is poorly defended. You set out to outline Chomsky's long and engaging political life just so that you can say that it influenced his linguistic theory. Chomsky himself have said many times that there is no clear correlation between his linguistic work and political activism. This is not new. He has been asked this in a lot of talks.

with this said, there were attempts at bridging the creativity exhibited by linguistic behavior and that of political freedom, such as in his "Language and Freedom" book, but this happen way after he established his theory.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#66

Post by jal278 »

VikingCoder wrote: post #62

I picture Chomsky as Kepler, trying to build orbits out of Platonic solids. Until Kepler had access to Brahe's data, he was not going to be able to come up with his theories of planetary motions. Worse than that, the laws of planetary motion present a simplistic view of the universe: what happens when a bunch of small objects orbit a very massive object. I think they wouldn't help you out at all, in trying to underst…

But there is a WHY associated with F= G* m1 * m2 / r^2 -- an explanation for why gravity exists, which although isn't yet unified with quantum mechanics, is that mass deforms space. The statistical formula is cool on its own right though too of course -- because of its elegance.

However, a concise formula without an explanation is less interesting, and an enormous statistical model without any elegance or insight (perhaps driven by big-data) is less interesting still, although it may have practical applications.

There may be general principles to intelligence, how it evolved through natural selection, and what in general will make for an intelligent robot. These are some of the most interesting unmade discoveries. But it isn't a sharp dichotomy between "Chomsky" and "Norvig," although that is one way to frame the question.

We should explore all reasonable approaches to AI and make sure one method is not dominating the others, but otherwise let everyone explore what they will. Who really knows what actually will lead to AI?

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#67

Post by bwanab »

phaedrus wrote: post #40

I spent about ten years working on Markov based chat programs. I gave up on themwhen I realized that no matter how sophisticated your statistical model it will never be more than a statistical analysis of text, unless it includes some rich rule based model of mental processes and mental objects. It may be that such a model of mental processes must itself be fuzzy and probabilistic, but it must exist. Therefore I come…

It strikes me that the theories are not mutually incompatible at all. They have very different purposes. Chomsky is trying to understand meaning and intelligence at a deep level. Norvig is trying to build models that help people right now (and incidently, help his company to make more money). Any new insights from either path will help refine the other.

So it sounds like what you decided was that you wanted to explore one path and not the other. Nothing wrong with that, but it's a very different statement.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#68

Post by nessus42 »

mootothemax wrote: post #8

Isn't this basically an argument over John Searle's Chinese Room thought experiment? It supposes that there is a program that gives a computer the ability to carry on an intelligent conversation in written Chinese. If the program is given to someone who speaks only English to execute the instructions of the program by hand, then in theory, the English speaker would also be able to carry on a conversation in written C…

> Isn't this basically an argument over John Searle's Chinese Room thought experiment?

No, this debate is completely and utterly different from Searle's Chinese Room argument! Searle's argument is a philosophical one for the assertion that a computer could never be a person. He concludes that this is true even if we were to eventually believe that we completely understand intelligence in the manner than Chomsky is lobbying for, and then fully implement that understanding in a computer.

For Searle, no amount of understanding of intelligence in any form will ever let us make an intelligent computer. For Searle, intelligent beings must be made out of flesh and bone. Or at least not out of anything digital and computer-like.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#69

Post by fat_clown »

It is an interesting debate, though I think it's being shone in the wrong light.

According to the article, it almost sounds like Chomsky believes a statistical approach to AI is a disservice to the field. The point he's missing is that research in statistical based AI is just that - statistics research.

Chomsky and Norvig deal in two different fields, which happen to have similar applications. Norvig does research in statistical and machine based learning. Success in this field comes from a new model that can make more accurate predictions, or a proof that it is impossible to make valid predictions about X with only Y as input. Applications of this field include technologies which rival AI systems as envisioned by Chomsky, but the essential point is that this field focuses on statistics research, not AI research.

Chomsky is wrong in dismissing this as a disservice. I do agree with his main point, that AI research and knowledge is not necessarily furthered by statistics research, but that is simply because they are different beasts entirely.

Maybe one day, when the biology has caught up with us and we have a solid understanding of the brain, will we be able to create a highly intelligent computer. Until then, statistics research is most likely to yield fruitful results.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#70

Post by icandoitbetter »

Jun8 wrote: post #43

OK, let me start with two facts, one objective, one personal: (i) Noam Chomsky is a genius with many contributions to linguistics and computer science (ii) I think his overall influence had been damaging to linguistics. Here's a summary of Chomsky's career in layman's terms: As everyone knows, Chomsky first came to prominence with his critique of Skinner (who, as everyone also knows, was a total psycho). He pretty mu…

> I detest politically motivated scientific theories

Then you detest every scientific theory ever devised.

Post reply on HN