Noam Chomsky on Where Artificial Intelligence Went Wrong • Hacker News Board

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#161

Post by mturmon » Fri, Nov 02, 2012, 5:23 PM UTC

confluence wrote: post #103

ilaksh wrote: post #83

The field is now called AGI. It isn't mentioned in this article. Everyone seems to be ignoring the whole field of AGI (artificial general intelligence). Or maybe they truly are ignorant of it. Anyway, suffice to say, AI and AGI didn't stop progressing, and Chomsky is no longer any sort of expert in those fields. Even Norvig isn't up to speed on the most advanced approaches to AGI, but at least he enters the same room…

I've heard of OpenCog before and it, along with the Singularity crowd gives me a weird amateur, bullshitty, vague, generalist feeling that Noam Chomsky does. Basically - where's the beef? What has been done by either crowd apart from taking credit from those who do things in the actual industry/real world? My fundamental aversion to both OpenCog and the entire Singularity crowd is a) their statements are so general a…

As a point of reference, here's the agenda for the most recent AGI conference:

http://agi-conf.org/2011/conference-schedule/

And, just for comparison, here's the agenda for the most recent ICML conference:

http://icml.cc/2012/schedule/

To me, the AGI conference seems to have a much higher ratio of "speculative ideas"/"technical results" talks. Also to me, this pretty much justifies the "all talk - no walk" assessment.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#162

Post by gavanwoolery » Fri, Nov 02, 2012, 10:37 PM UTC

adrianhoward wrote: post #139

gavanwoolery wrote: post #39

While I respect (some of Chomsky's work) it is amazing to me that he thinks language is something more than mathematics/statistics. Language is math, it is basically an advanced form of discreet mathematics. We can reproduce virtually anything on a computer, and it is no more "shallow" than artificial light from a lightbulb is "artificial." There is no magic going on, we are biological computers, walking number crunc…

it is amazing to me that he thinks language is something more than mathematics/statistics He doesn't think that. Look at his linguistics work - google Chomsky Hierarchy. He very much does not think that it's magic. All he's saying is that some kinds of statistical modeling - while stupidly useful and practical - don't give us a lot of explanatory power. He's, metaphorically, complaining about folk who are happy using…

Well aware of his work, but I realize I probably misread the article. :) I think I just expanded statistical modeling to encompass more fields than I meant to (you can, in fact, employ statistical methods to augment "deeper" systems like automatically developing rule systems for propositional logic).

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#163

Post by snowwrestler » Fri, Nov 02, 2012, 11:41 PM UTC

AndyNemmity wrote: post #25

zepolud wrote: post #19

I've always considered Norvig's position pretty much self-evident for anybody who has dealt with real world problems. At the same time, Chomsky is still way more interesting to read, even if he's wrong. And he's wrong mostly for assuming that unsupervised learning has to be the end result when in fact it could be an intermediate step to more refined symbolic theories. In any case, this kind of antagonism between the…

By real world problems you're talking about engineering and not science. Chomsky says Norvig's position is great for engineering, just not science. The whole disagreement stems from the idea that this will help us with a scientific understanding of language or not. The burden is on Norvig's side to prove it, and it hasn't.

I think the phrase "scientific understanding" is essentially qualitative, meaning that humans can disagree about what we "understand" and how "scientific" it is, and there's no objective way to adjudicate that fight.

How many physicists really believe they fully understand quantum mechanics? The theory is probablistic and strange but produces very accurate predictions.

The true measure of science is matching observations to hypotheses, and in that respect the approach that Norvig defends has demonstrated success. Google's language tools work well much of the time. Watson beat Ken Jennings.

Other branches of science are beginning to make more use of "big data" approaches as well. A friend doing post-doctoral research on evolution spends most of his time behind a laptop coding against big sets of digitized genetic information.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#164

Post by corporalagumbo » Fri, Nov 02, 2012, 11:47 PM UTC

confluence wrote: post #119

corporalagumbo wrote: post #112

Earlier quoted context omitted.

Chomsky's expertise is in linguistics and political analysis. Stephen Pinker's The Language Instinct is a good, readable introduction to some of Chomsky's work (and the wider field to which he is pivotal.) Chomsky's Manufacturing Consent is probably his classic work of political analysis. http://en.wikipedia.org/wiki/Noam_Chomsky_bibliography He's no quack.

You know in the soft sciences everyone is a quack because fundamentally they don't practice - wait for it - science. Science stops false connections by correctly attributing cause to its respective effect. Social sciences do not. For all intents and purposes, the vast majority of social science is either unreproducible, vague, mixing correlation with causation, uses dependent variables, poorly reasoned, statistical q…

Angry much? Have you actually read Chomsky, or are you just taking snippets from Wikipedia pages and saying told-you-so? Perhaps you should try reading Manufacturing Consent, it's a very careful and thorough work of analysis and not nearly as bleedingly obvious as you try and portray it.

One point: Sampson's criticisms about linguists producing post-hoc descriptions could just as easily have been (and were, I believe) applied to Newton's theories. Good science includes mapping and describing phenomena.

Another point: negative feedback on errors is not enough to account for the explosive speed of language acquisition in children. Not to say that this sort of feedback doesn't occur, or isn't useful, but it only really is used when children learn exceptions (I.e. irregular verb forms in English) or vocabulary (and even much of vocabulary is rule-generated.) Basic language rules are encoded, and children's brains only require minimal stimulus to record the specific settings of the rules for the language they are learning.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#165

Post by cuspy » Sat, Nov 03, 2012, 5:39 AM UTC

Those who argue that Chomsky singularly "pioneered" or "revolutionized" the study of formal language should thoroughly read the book "Linguistics and the Formal Sciences" by Marcus Tomalin. It is a great historical account of the development of this particular strain of formal linguistic study. Knowing more about the intellectual environment and his predecessors and contemporaries helps to erode the mythology of Chomsky as the sole revolutionary catalyst in the development of formal language theory. In fact the major principles of his classic theory of syntax can be thought of as fairly incremental developments from previous work. Many of the specific claims he is well known for had been made by others before.

As for his contributions to cognitive science, I think one side of the field simply feels that he is clinging to some outmoded notions of what Bayesian modeling can achieve in terms of explanatory power.

As a counterpoint, EVERYONE should read Andy Clark's beautifully written BBS paper "Whatever Next? Predictive Brains, Situated Agents, and the Future of Cognitive Science."

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#166

Post by bengoertzel » Sat, Nov 03, 2012, 10:04 AM UTC

confluence wrote: post #103

ilaksh wrote: post #83

The field is now called AGI. It isn't mentioned in this article. Everyone seems to be ignoring the whole field of AGI (artificial general intelligence). Or maybe they truly are ignorant of it. Anyway, suffice to say, AI and AGI didn't stop progressing, and Chomsky is no longer any sort of expert in those fields. Even Norvig isn't up to speed on the most advanced approaches to AGI, but at least he enters the same room…

I've heard of OpenCog before and it, along with the Singularity crowd gives me a weird amateur, bullshitty, vague, generalist feeling that Noam Chomsky does. Basically - where's the beef? What has been done by either crowd apart from taking credit from those who do things in the actual industry/real world? My fundamental aversion to both OpenCog and the entire Singularity crowd is a) their statements are so general a…

Hi, this is Ben Goertzel, the chief founder of the OpenCog AGI-focused software project and of the AGI conference series.

Comparing Google Search and IBM Watson to OpenCog and other early-stage research efforts is silly. Google Search and IBM Watson have taken fairly mature technologies, pioneered by others over decades of research, and productized them fantastically. OpenCog is a research project and is aimed at breaking fundamentally new research ground, not at productizing and scaling-up technologies already basically described in the academic literature.

Lecturing is a very small percentage of what those of us involved with OpenCog do. We are building complex software and developing associated theory. Indeed parts of our approach are speculative, and founded in intuition alongside math and empirics. That's how early-stage research often goes.

Of course you can trash all early-stage research as not having results yet. And the majority of early-stage research will fail, probably making you tend to feel vindicated and high and mighty in your skepticism ;p .... But then, a certain percentage of early-stage research will succeed, because of researchers having the guts to follow their intuitions in spite of the ceaseless tedious sniping of folks like you ;p ...

- Ben Goertzel

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#167

Post by bengoertzel » Sat, Nov 03, 2012, 10:08 AM UTC

mturmon wrote: post #161

confluence wrote: post #103

Earlier quoted context omitted.

I've heard of OpenCog before and it, along with the Singularity crowd gives me a weird amateur, bullshitty, vague, generalist feeling that Noam Chomsky does. Basically - where's the beef? What has been done by either crowd apart from taking credit from those who do things in the actual industry/real world? My fundamental aversion to both OpenCog and the entire Singularity crowd is a) their statements are so general a…

As a point of reference, here's the agenda for the most recent AGI conference: http://agi-conf.org/2011/conference-schedule/ And, just for comparison, here's the agenda for the most recent ICML conference: http://icml.cc/2012/schedule/ To me, the AGI conference seems to have a much higher ratio of "speculative ideas"/"technical results" talks. Also to me, this pretty much justifies the "all talk - no walk" assessment…

This is Ben Goertzel, chief founder of the AGI conference series.

You are correct that the AGI conferences have a higher ratio of "speculative ideas"/"technical results" to ICML. This is intentional and I belief appropriate -- because AGI is at an earlier stage of development than machine learning, and because it's qualitatively different in character than machine learning.

Machine learning (in the sense that the term is now typically used, i.e. supervised classification, clustering, data minign, etc.) can be approached mainly via a narrowly disciplinary approach. Some cross-disciplinary ideas have proved valuable, e.g. GAs and neural nets, but the cross-disciplinary ideas there have quickly been "computer science ized"...

OTOH, I think AGI is inherently more complex and multifarious than ML as currently conceived, and hence requires more "out of the box" and freely multi-disciplinary thinking.

I think that in 10-15 years, when the AGI field is much more mature, the conferences will seem a bit more like ML conferences in terms of the percentage of papers reporting strong technical results. BUT, they will never seem as narrowly disciplinary as ML conferences, because AGI is a different sort of pursuit...

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#168

Post by bengoertzel » Sat, Nov 03, 2012, 10:17 AM UTC

joe_the_user wrote: post #115

ilaksh wrote: post #105

Earlier quoted context omitted.

So I figured it out. Basically, they take the idea of AGI seriously, and actually consider and talk about the repercussions, and therefore you dismiss them and their ideas as fringe and not worth investigating. I know that, because if you had investigated at all, you would see that all of those projects had really interesting results and these people are not being vague and hand-waving. Not all of those projects I li…

I'm sympathetic to both Chomsky and Open Cog's aims. I know Chomsky is a serious scientist with considerable accomplishment. I have seen totally loony stuff in videos of AGI conferences (Tachyons and stuff). Open Cog may be better than that. But it hasn't proved that it is better than that. The 1970-80's AI involved the Chomskyan paradigm of "draw up a naive design of the mind and/or brain and implement it". That fai…

This is Ben Goertzel...

AGI is mainstream science, these days. The keynote of the 2012 AAAI conference (the major mainstream AI research conference each year), by the President of AAAI, was largely about how the time has come for the AI field to refocus on human-level AI. He didn't use the term "AGI" but that was the crux of it.

The "AI winter" is over. Maybe another will come, but I doubt it.

What's different from 20 years ago? Hardware is way better. The Internet is way richer in data, and faster. Software libraries are way better. Our understanding of cognitive and neural science is way stronger. These factors conspire to make now a much better time to approach the AGI problem.

As for my own AGI research lacking anything new, IMO you think this because you are looking for the wrong sort of new thing. You're looking for some funky new algorithm or knowledge structure or something like that. But what's most novel in OpenCog is the mode of organization and interaction of the components, and the emergent structures associated with them. I realize it's a stretch for most folks to realize that the novel ingredients needed to make AGI lie in the domain of systemic organizational principles and emergent networks rather than novel algorithms, data structures or circuits -- but so it goes. It wouldn't be the first time that the mass of people were looking for the wrong kind of innovation, hmm?

Regarding tachyons in videos of AGI conferences, could you provide a reference? AGI conference talks are all based on refereed papers published by major scientific publishers. Some papers are stronger than others, but there's no quackery there.... (There have been "Future of AGI" workshops associated with the AGI conferences, which have had some freer-ranging speculative discussions in them; could you be referring to a comment an audience participant made in a discussion there?)

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#169

Post by jmmcd » Sat, Nov 03, 2012, 8:41 PM UTC

> if success is defined as getting a fair approximation to a mass of chaotic unanalyzed data, then it's way better to do it this way than to do it the way the physicists do, you know, no thought experiments about frictionless planes and so on and so forth

This is factually incorrect! For physicists, those thought experiments are absolutely essential, and we would need many, many more orders of magnitude of statistical processing of video signals in order to get close to the real-world-useful physical predictions that we arrive at through thought experiments, equations, and so on. The contrast that Chomsky is missing is that for language, the statistical processing is amazingly successful, and the thought experiment style of investigation, while productive, has not been shown useful in real world tasks like translation.

For those arguing against Chomsky, none of the above means that we should abandon a theory-driven or symbolic approach to language.

If Chomsky and his opponents would just recognise that they have different goals (not just different ways of approaching the same goal), we wouldn't have to have this same argument every few months.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#170

Post by mturmon » Sat, Nov 03, 2012, 9:53 PM UTC

bengoertzel wrote: post #167

mturmon wrote: post #161

Earlier quoted context omitted.

As a point of reference, here's the agenda for the most recent AGI conference: http://agi-conf.org/2011/conference-schedule/ And, just for comparison, here's the agenda for the most recent ICML conference: http://icml.cc/2012/schedule/ To me, the AGI conference seems to have a much higher ratio of "speculative ideas"/"technical results" talks. Also to me, this pretty much justifies the "all talk - no walk" assessment…

This is Ben Goertzel, chief founder of the AGI conference series. You are correct that the AGI conferences have a higher ratio of "speculative ideas"/"technical results" to ICML. This is intentional and I belief appropriate -- because AGI is at an earlier stage of development than machine learning, and because it's qualitatively different in character than machine learning. Machine learning (in the sense that the ter…

Thanks for the kind reply. I said ICML, but NIPS would have been a better point of reference -- since it was originally conceived as a cross-disciplinary enterprise. The NIPS TOC looks like this:

http://nips.djvuzone.org/nipsxx-toc.html

which indicates it's possible to have a selection of papers both technically sharp and interdisciplinary. We should all be so lucky to attract such a set of papers.

I'm reminded of the 1958 editorial by Peter Elias in the IEEE Information Theory Transactions ("Two Famous Papers"): http://oikosjournal.files.wordpress.com/2011/09/elias1958ire...

I sincerely wish you, your conference, and your research enterprise the best.