Live data from Hacker News

Noam Chomsky on Where Artificial Intelligence Went Wrong

theatlantic.com

81–90 of 184 posts

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#81

Chomsky is right up there with Minsky in being part of the problem. His ideas about language being part of the genome are fanciful nonsense. Skinner produce reams of reproducible empirical observations of behavior which are, today, critical to the evaluation of the performance of AIs. Chomsky has produced interesting theories, but mostly derailed linguistics on the basis of the argument 'language is complicated, some…

I hope you're kidding about his contributions to political debate. Chomsky has the maximalist attitude that represents everything that is wrong with political debate today. His hatred of U.S. foreign policy is so extreme that he will defend absolutely everyone the U.S. opposes which means occasionally defending tyrants and denying genocide.

I'm amused by the idea that Chomsky is somehow some kind of dunce when it comes to science but a brilliant thinker when it comes to politics. At least with either claim individually, I can conceive of who might say that, even while thinking they're radically wrong (no pun intended).

But the idea that Chomsky's political "contributions" somehow dwarf his contributions to science? That seriously floors me.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#82

Chomsky is right up there with Minsky in being part of the problem. His ideas about language being part of the genome are fanciful nonsense. Skinner produce reams of reproducible empirical observations of behavior which are, today, critical to the evaluation of the performance of AIs. Chomsky has produced interesting theories, but mostly derailed linguistics on the basis of the argument 'language is complicated, some…

I hope you're kidding about his contributions to political debate. Chomsky has the maximalist attitude that represents everything that is wrong with political debate today. His hatred of U.S. foreign policy is so extreme that he will defend absolutely everyone the U.S. opposes which means occasionally defending tyrants and denying genocide.

> His hatred of U.S. foreign policy

Sadly enough for (U.S. foreign policy and its supporters) he actually supports his claims very well with sources and footnotes, and if you read through this works, you might find that perhaps there is a reason others (who don't just watch Fox News) don't agree with said policy, and also that somehow Americans in certain parts of the world are not "hated because of our freedoms". There are other reasons.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#83
The field is now called AGI. It isn't mentioned in this article. Everyone seems to be ignoring the whole field of AGI (artificial general intelligence). Or maybe they truly are ignorant of it.

Anyway, suffice to say, AI and AGI didn't stop progressing, and Chomsky is no longer any sort of expert in those fields.

Even Norvig isn't up to speed on the most advanced approaches to AGI, but at least he enters the same room with people who are aware of the field. For example, he gave a talk at the recent Singularity Summit.

The Fifth Conference on Artificial General Intelligence is going to be in Oxford in December. http://agi-conference.org/2012/

Here is some information for people who are interested in pertinent ideas related to AGI.

http://www.amazon.com/How-Create-Mind-Thought-Revealed/dp/06...

http://opencog.org/theory/

>OpenCog is a diverse assemblage of cognitive algorithms, each embodying their own innovations — but what makes the overall architecture powerful is its careful adherence to the principle of cognitive synergy.

>The human brain consists of a host of subsystems carrying out particular tasks — some more specialized, some more general in nature — and connected together in a manner enabling them to (usually) synergetically assist rather than work against each other.

http://wiki.opencog.org/w/Probabilistic_Logic_Networks

> PLN is a novel conceptual, mathematical and computational approach to uncertain inference. In order to carry out effective reasoning in real-world circumstances, AI software must robustly handle uncertainty. However, previous approaches to uncertain inference do not have the breadth of scope required to provide an integrated treatment of the disparate forms of cognitively critical uncertainty as they manifest themselves within the various forms of pragmatic inference. Going beyond prior probabilistic approaches to uncertain inference, PLN is able to encompass within uncertain logic such ideas as induction, abduction, analogy, fuzziness and speculation, and reasoning about time and causality.

http://wiki.opencog.org/w/AtomSpace

Conceptually, knowledge in OpenCog is stored within large [weighted, labeled] hypergraphs with nodes and links linked together to represent knowledge. This is done on two levels: Information primitives are symbolized in individual or small sets of nodes/links, and patterns of relationships or activity found in [potentially] overlapping and nesting networks of nodes and links. (OCP tutorial log #2).

http://www.izhikevich.org/publications/large-scale_model_of_...

Large-Scale Model of Mammalian Thalamocortical Systems

> The understanding of the structural and dynamic complexity of mammalian brains is greatly facilitated by computer simulations. We present here a detailed large-scale thalamocortical model based on experimental measures in several mammalian species. The model spans three anatomical scales. (i) It is based on global (white-matter) thalamocortical anatomy obtained by means of diffusion tensor imaging (DTI) of a human brain. (ii) It includes multiple thalamic nuclei and six-layered cortical microcircuitry based on in vitro labeling and three-dimensional reconstruction of single neurons of cat visual cortex. (iii) It has 22 basic types of neurons with appropriate laminar distribution of their branching dendritic trees. The model simulates one million multicompartmental spiking neurons calibrated to reproduce known types of responses recorded in vitro in rats. It has almost half a billion synapses with appropriate receptor kinetics, short-term plasticity, and long-term dendritic spike-timing-dependent synaptic plasticity (dendritic STDP). The model exhibits behavioral regimes of normal brain activity that were not explicitly built-in but emerged spontaneously as the result of interactions among anatomical and dynamic processes. We describe spontaneous activity, sensitivity to changes in individual neurons, emergence of waves and rhythms, and functional connectivity on different scales.

http://www.sciencebytes.org/2011/05/03/blueprint-for-the-bra...

Essentials of General Intelligence: The direct path to AGI

http://www.adaptiveai.com/RealAI_chap_ver2c.htm

>General intelligence, as described above, demands a number of irreducible features and capabilities. In order to proactively accumulate knowledge from various (and/ or changing) environments, it requires:

>1. Senses to obtain features from ‘the world’ (virtual or actual),

>2. A coherent means for storing knowledge obtained this way, and

>3. Adaptive output/ actuation mechanisms (both static and dynamic).

>Such knowledge also needs to be automatically adjusted and updated on an ongoing basis; new knowledge must be appropriately related to existing data. Furthermore, perceived entities/ patterns must be stored in a way that facilitates concept formation and generalization. An effective way to represent complex feature relationships is through vector encoding (Churchland 1995).

>Any practical applications of AGI (and certainly any real-time uses) must inherently be able to process temporal data as patterns in time – not just as static patterns with a time dimension. Furthermore, AGIs must cope with data from different sense probes (e.g., visual, auditory, and data), and deal with such attributes as: noisy, scalar, unreliable, incomplete, multi-dimensional (both space/ time dimensional, and having a large number of simultaneous features), etc. Fuzzy pattern matching helps deal with pattern variability and noise.

>Another essential requirement of general intelligence is to cope with an overabundance of data. Reality presents massively more features and detail than is (contextually) relevant, or that can be usefully processed. This is why the system needs to have some control over what input data is selected for analysis and learning – both in terms of which data, and also the degree of detail. Senses (‘probes’) are needed not only for selection and focus, but also in order to ground concepts – to give them (reality-based) meaning.

http://en.wikipedia.org/wiki/Hierarchical_temporal_memory

> A typical HTM network is a tree-shaped hierarchy of levels that are composed of smaller elements called nodes or columns. A single level in the hierarchy is also called a region. Higher hierarchy levels often have fewer nodes and therefore less spacial resolvability. Higher hierarchy levels can reuse patterns learned at the lower levels by combining them to memorize more complex patterns.

> Each HTM node has the same basic functionality. In learning and inference modes; sensory data comes into the bottom level nodes. In generation mode; the bottom level nodes output the generated pattern of a given category. The top level usually has a single node that stores the most general categories (concepts) which determine, or are determined by, smaller concepts in the lower levels which are more restricted in time and space. When in inference mode; a node in each level interprets information coming in from its child nodes in the lower level as probabilities of the categories it has in memory.

>Each HTM region learns by identifying and memorizing spatial patterns - combinations of input bits that often occur at the same time. It then identifies temporal sequences of spatial patterns that are likely to occur one after another.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#84
post #21

Earlier quoted context omitted.

I did not reference linguistics literature, but philosophy literature (a distinction worth making because they are approaching the problem at different levels). I've never been good at the 'language problem' elevator speech, but how is it possible that we can capture the full power of language while using language to describe it? Language is a technology that allows for the production of concepts like 'probability' o…

You're asking if I believe in qualia, and the answer is no. There are firing neurons and that's it. The great variety of ways in which neurons can fire, and the great variety of experiences that shape how neurons fire combine to form an exquisite set of possible firing patterns (this is literally what makes me me and you you) but ultimately, to mis-use Gertrude Stein's famous phrase, 'there is no there there.'

It wasn't at all clear that that is what he was asking you. And you need to qualify the sense in which you "don't believe" in qualia. You don't believe that consciousness has phenomenal properties? Qualia certainly exist in some sense.

From what it sounds like, you are just dismissing compelling philosophical issues because it frustrates your beliefs.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#85
post #65

Earlier quoted context omitted.

Exactly. Well said, and clearly articulates the point here. What I don't get is, what Chomsky is saying is all together the standard, and yet people are insulting him for what is all together a very simple idea you expressed plainly. Yes, NLP systems are going to have many engineering uses, and Chomsky agrees. Are they going to help in the true scientific understanding of the systems? It's unlikely. It's likely to be…

Yes, NLP systems are going to have many engineering uses, and Chomsky agrees. Are they going to help in the true scientific understanding of the systems? So, what we have is (1) the engineering / statistical modelling / machine learning approach, and (2) the deep theoretical "Chomsky approach". Chomsky despises, maybe rightly so, the engineering approach because it only provides tools that work, approximately, in pra…

> Why has all the practical progress come from the engineers?

I see what you are saying and actually it is a good point. Where are the robots built on Chomsky's theory? A very valid question. I don't know the answer to it, Chomsky doesn't either. But I think what you mean by practical progress isn't what he mean progress. That is his point.

You have to see where he is coming from. He is an academic his ultimate goal is to understand how things work. Training a set of neurons with input data and ending up perhaps with millions activation weights in the end is not helping that goal even if this new machine can play chess, make coffee and drive you to work. I think that is his take on it.

I say we need both. There is no reason to not strive for both. There is not reason to turn all radical and start burning books and claim one approach should completely replace the other. I hope we one day find (or find that we can't find) a good explanatory model for meaning, language, learning, personality, or conscience, but in the meantime I enjoy playing chess with my computer, and I hope pretty soon I'll have my car drive me to work by itself.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#86
post #83

The field is now called AGI. It isn't mentioned in this article. Everyone seems to be ignoring the whole field of AGI (artificial general intelligence). Or maybe they truly are ignorant of it. Anyway, suffice to say, AI and AGI didn't stop progressing, and Chomsky is no longer any sort of expert in those fields. Even Norvig isn't up to speed on the most advanced approaches to AGI, but at least he enters the same room…

> The field is now called AGI.

No it's not.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#87
post #38
post #17

Earlier quoted context omitted.

What could 'meaning' possibly be other than probabilistic associations of sounds and concepts?

Your observation is so vague and general to the point of being rather meaningless. Almost every physical theory is described by an underlying mapping between inputs. The interesting point is the expression power of your model. : to take an example I am somehow familiar with, current large vocabulary speech recognizers have millions of parameters. They work relatively well, but they are very difficult to interpret, an…

Do you have any reason to suspect this isn't how the brain works? Maybe language isn't a small set of high level rules. Why should we suspect it to be? The probablistic models seem to be very similar to how real people actually learn informal language. Formal languages of course have high-level rules, and these are well modelled algorithmically.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#88
post #17

Earlier quoted context omitted.

What could 'meaning' possibly be other than probabilistic associations of sounds and concepts?

The idea that meaning consists of associations is extremely primitive. It works for concrete nouns and verbs but it quickly fails as things get more complex. Language is used to refer to refer to abstract things, imaginary things, counterfactual situations, etc. And even if you do arrive at a series of concepts using associations, you have to understand how they are supposed to combine, even for completely novel sent…

Please argue that there is nothing to associate with.

Why is a real observation from your senses more privileged inside your brain that a random well-formed value by a (hypothetical) random number generator neuron?

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#89

Chomsky is right up there with Minsky in being part of the problem. His ideas about language being part of the genome are fanciful nonsense. Skinner produce reams of reproducible empirical observations of behavior which are, today, critical to the evaluation of the performance of AIs. Chomsky has produced interesting theories, but mostly derailed linguistics on the basis of the argument 'language is complicated, some…

I made similar comments when the Chomsky vs. Norvig discussion came up on HN some time ago and got interesting replies: http://news.ycombinator.com/item?id=4291327.

Re: Noam Chomsky on Where Artificial Intelligence Went Wrong

#90
post #34

Earlier quoted context omitted.

I don't see any mysticism in Chomsky's approach. You seem to agree with Norvig that doing massive data analysis on language will come to a scientific understanding, which would be a first in science. Chomsky doesn't. If anything, Chomsky is grounded in reality, and Norvig and AI researchers are grounded in hope that this way of mapping out something will create meaningful understanding of the system.

This reminds me a bit of what scientists in other fields refer to as "empirical equations", which are equations fit from data without any particular theoretical backing or reason to believe that their components are a good model of reality. They're useful in that they may predict observations well, especially over a specific range of observables, but they don't necessarily give us an understanding of what's going on.…

Why do people keeping saying that probabilistic models do not provide understanding? We flew to Mars on a few simple principles applied to massive amounts of data. Comets aren't following sophisticated orbital plans, just F=ma.

Maybe (probably) all the sophistication of natural language is an emergent property of the pile on of lots of little similarly-shaped details like atoms. Sure, high level rules are nice approximations that satisfy our human craving for patterns, but that doesn't mean those patterns are how the brain really works, quite the opposite in fact.

High level models are illustrations, useful for game programmers and artists to efficiently create simulations and plausible imaginary creations. Low level models are how things actually work.

Post reply on HN