Live data from Hacker News

Norvig vs. Chomsky and the Fight for the Future of AI

tor.com

101–110 of 152 posts

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#101
AI research, including speech recognition and machine vision, are currently ENGINEERING disciplines trying to make artifacts that do interesting things. Success is an artifact that works.

Several basic science disciplines are trying to understand how brains work. There is mostly tremendous amounts of experimental facts, difficult to put together, and some theory and modelling to go with it.

Norvig would be confused if he thinks that engineering AI systems automatically counts as models useful for understanding the brain. If there is application to understanding brains it is a welcome accident. It happens that there are signals in basal ganglia that look like the temporal difference error signal from reinforcement learning. So maybe RL research can help understand some brain circuitry in that case.

But in general the engineers are trying to get stuff to work, and they are deluded if they think they are simultaneously making progress in understanding how brains work.

EDIT:

For example: why does speech recognition use hidden markov models and N-gram language models? Because they're the best model of how brains understand speech? No! Not at all. HMMs and N-gram models are above all computationally tractable. Easy to implement, not too slow to run.

We have algorithms (such as baum-welch and N-gram smoothing techniques) to get them work work well in engineering applications. Nothing more. Might they help us understand brains? Maybe, but not at all necessarily so.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#102
post #99

It's a little bit frustrating to read a rehash of an argument that was cutting edge maybe back in the late 90s, especially one that is so poorly written, and framed as a battle between two intellectuals. Chomsky's past his heyday. He has been seminal in his field, but he's no longer doing research which pushes at the boundaries of our understanding of language, how to model it, or what the fundamental nature of langu…

Non-Chomskian linguists? I was under the impression that, post-Chomsky, linguistics was defined as Chomskian; everyone else had left the field for whatever related discipline most closely matched what they wanted to do.

I mean... there are core things at the basis of what everyone agrees on in linguistics (language is structured, learned, and there is a defined syllabary of sounds that humans make and use in language, etc), but Chomsky's grand unified theory of language, and his ideas about how those mechanics function are not universally agreed upon.

Even within syntax, which is really the sort of core of what Chomsky has been interested in, there are other formalisms to represent syntax which differ from Chomsky's theoretical framework. You can read more about Chomskian transformational grammar on wikipedia with links off to other sorts of formalisms as well: http://en.wikipedia.org/wiki/Transformational_grammar

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#103

I spent about ten years working on Markov based chat programs. I gave up on themwhen I realized that no matter how sophisticated your statistical model it will never be more than a statistical analysis of text, unless it includes some rich rule based model of mental processes and mental objects. It may be that such a model of mental processes must itself be fuzzy and probabilistic, but it must exist. Therefore I come…

There are two possible ways to invent this kind of theory, the way I see it:

1.) Measure data, then make an educated guess. 2.) Make an uneducated guess.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#104
There must be some analogies made to the much smaller field of chess computer programs.

From 1950s to about 1980 or so it was thought that the best computer chess program would approximate the way a human would think about the game. Botvinnik in particular was adamant that such an approach would be the right one.

However, most of the progress was made through brute force. Modern chess programs select moves in a way that is far removed from the way a good chessplayer selects moves, yet they can now produce games that seem very "uncomputer" like and "human".

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#105
Historically, AI has been divided into two related but different approaches. "Strong" AI is interested in understanding and creating Minds; figuring out what intelligence is, how it works, how we do it, and how it could be done in general. "Weak" AI is interested in doing things that couldn't be done before; things that we do not have good algorithms for, or don't have any algorithms at all.

Those two are not opposed. Any advance on either side helps the other. In this argument, Norvig is representing an extreme version of weak AI since he seems to be arguing that it's possible that statistical methods are all there is. (I suspect that he isn't actually making that argument, though, but that strong AI's models are currently too simplistic to capture what statistical approaches can do.) Chomsky, on the other hand, seems to be caricaturing strong AI by saying that anything that doesn't directly shed light on the Grand Theory is worthless.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#106
post #100
post #97

Earlier quoted context omitted.

I'm completely ignorant here - is it widely established that there's no link between how we think about things versus the languages we speak ? It seems intuitive that our ability to conceive of concepts would be dependent on having/creating language that can describe them, even in our own minds, but I have no idea how anyone could really know one way or the other.

As you point out, this is a hard thing to test. Add to this the fact that the question may be a sensitive one, similar to differences between men and women. How can one go about testing the effect of language on thinking? Consider this example: English has an explicit grammatical structure for counterfactuals (CFs), e.g. "If I were a rich man ..." whereas some languages, e.g. Chinese (Mandarin) do not (they do have s…

so you're saying, it's not that there's no link, just that it's hard to pin down, and any real conclusions are inherently dangerous from a political/anthropological stance. that makes sense as I've heard from others that the field of anthropology has been in a semi-permanent state of apologism for eurocentric thought for the past thirty years.

I guess some forms of truth can't be released until society has fully prepared itself structurally for them (i.e., a society that could learn of inherent differences in thought based on language, without starting a new caste system based on that).

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#107
post #57

Earlier quoted context omitted.

I have read most of the language instinct. I get why people find it compelling, but that has little to do with being accurate. My point was his style tends to be convincing vs. his actual evidence being compelling. PS: Think of it like this A -> B, B -> C therefore A -> C is all well in good most of the time . A -> B .... Y -> Z therefore A -> Z only really works with math, build a chain that long and it's unlikely f…

And I don't find your point in the least bit compelling. Unless you're a leading expert in the field as he is, your points mean jack squat to me. And since I'm not making the argument, it's not argument from authority to say his book is far more convincing than your assertions without evidence. You're trying to debate me about a book I recommended; you're an ass. Goodbye.

That seems overly rude. It's also a ridiculous appeal to authority at the same time.

If I where to attack the book I could say something like: "In chapter 2 'Chatterboxes' he states humans are the only animal that uses language which is complex issue by it's self. He goes further and says every group of humans in remote areas we encountered have had complex language. He then runs with that line of reasoning. However this ignores not just other animals that use simple forms of language but human ancestors that where close to us anatomically and probably also used language. If homo sapiens's ancestors also used language then you would expect the earliest humans to also use language therefore language would spread from it's origins to all those remote areas vs. being created from scratch in those remote areas."

Now, I could get 10 PHD's to say the same thing and use quotes etc, but what's important is the accuracy of the statement not who says it. http://lesswrong.com/lw/jl/what_is_evidence/

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#108

It is an interesting debate, though I think it's being shone in the wrong light. According to the article, it almost sounds like Chomsky believes a statistical approach to AI is a disservice to the field. The point he's missing is that research in statistical based AI is just that - statistics research. Chomsky and Norvig deal in two different fields, which happen to have similar applications. Norvig does research in…

Chomsky sees a threat to a politics-based academic hegemony, so he responds politically. If he hadn't made so many broadly quantified claims (i.e. had stuck to syntax as Norvig sticks to AI and machine learning) then those claims would not be in such jeopardy from other fields. Chomsky has always been a top-class warrior and this is more of the same.

heh.

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#109
post #100
post #97

Earlier quoted context omitted.

I'm completely ignorant here - is it widely established that there's no link between how we think about things versus the languages we speak ? It seems intuitive that our ability to conceive of concepts would be dependent on having/creating language that can describe them, even in our own minds, but I have no idea how anyone could really know one way or the other.

As you point out, this is a hard thing to test. Add to this the fact that the question may be a sensitive one, similar to differences between men and women. How can one go about testing the effect of language on thinking? Consider this example: English has an explicit grammatical structure for counterfactuals (CFs), e.g. "If I were a rich man ..." whereas some languages, e.g. Chinese (Mandarin) do not (they do have s…

I am no expert in this area, but I believe this is a fairly good example of the issue you are describing. According to the study, it appears as if the gender system used in some languages appears to bias individuals perception of the object.

http://public.wsu.edu/~fournier/Teaching/psych592/Readings/G...

Re: Norvig vs. Chomsky and the Fight for the Future of AI

#110

It's a little bit frustrating to read a rehash of an argument that was cutting edge maybe back in the late 90s, especially one that is so poorly written, and framed as a battle between two intellectuals. Chomsky's past his heyday. He has been seminal in his field, but he's no longer doing research which pushes at the boundaries of our understanding of language, how to model it, or what the fundamental nature of langu…

It's a rehash of http://norvig.com/chomsky.html and the talk by Chomsky that led to it (both recent and relevant). I can't wait to read the continuation of that article that Norvig promised in comments.
Post reply on HN