Building high-level features using large scale unsupervised learning • Hacker News Board

Re: Building high-level features using large scale unsupervised learning

#151

Post by _delirium » Fri, Jun 22, 2012, 4:22 PM UTC

chmike wrote: post #148

_delirium wrote: post #55

Earlier quoted context omitted.

I think much of this is overhyped as well, but I disagree that whether modeling human brain structure is relevant. The term "neural network" has historical baggage (responsible for some of the hype), but these days refers to a class of mathematical approaches with only historical connection to "neurons". Those can be interesting on their own for AI purposes, and imo accurate modeling of the human brian, while interes…

My use of modeling was indeed inappropriate. What I meant is the working principle of cortical neural networks. The artificial model would then be a kind of proof of concept. Regarding you other point, it is a matter of research strategy. I think that the path trying understanding the working principle of real cortical neural network is the shortest path to AI. My impression is that the other path which is to play ar…

I guess I take the opposite suggestion from the flight example: we succeeded in flying once we started focusing more on the physical/mathematical research of aerodynamics and lift, and less on attempting to mimic biology, in copying the biomechanics of bird wings. We ended up producing something that flies, but not in exactly the way that birds fly. That's what I tend to view as the better route for AI as well: instead of trying to copy the details of how a brain works, focus more on first-principles mathematical/logical principles of inference, whether they're symbolic ones (e.g. theorem-proving) or statistical ones (Bayesian networks, etc.).

Admittedly this is a big area of disagreement both within and outside the field.

Re: Building high-level features using large scale unsupervised learning

#152

Post by [deleted] » Fri, Jun 22, 2012, 4:27 PM UTC

JohnHaugeland wrote: post #135

redmoskito wrote: post #127

Earlier quoted context omitted.

I find both of your comments extremely condescending, both toward saalweachter and the authors of this article. 1. The fact that Claude Shannon succeeded in training a chess system has virtually no impact on sallweachter's claim that many AI results were overstated. 2. Certainly the press overstated them, which supports saalweachter's premise rather than weakening it. Even if the _implied_ claim was that _researchers…

"I find both of your comments extremely condescending" When a comment opens with a tone like this, I usually don't bother to respond, but I'll give you a chance, because you seem to have done a lot of honest mis-reading. To wit, it may be of value for you to inspect your own tone, if you find public condescention inappropriate. . "1. The fact that Claude Shannon succeeded in training a chess system has virtually no i…

[deleted]

Re: Building high-level features using large scale unsupervised learning

#153

Post by ehsanu1 » Fri, Jun 22, 2012, 4:37 PM UTC

SafeSituation wrote: post #129

mjwalshe wrote: post #64

I thought that the best so far was simulating 1/2 a mouse brain compextity at 1/2 speed.

Where can I find this study?

He might be referring to blue brain, though I recall that simulated 100k neurons from the neocortex (not quite half a mouse brain). The thing is that they simulated the physical processes in the synapses and more for a more accurate representation, so it's actually a lot cooler than it sounds!

Here's a link I googled up: http://bluebrain.epfl.ch/cms/lang/en/pid/56882

Re: Building high-level features using large scale unsupervised learning

#154

Post by Symmetry » Fri, Jun 22, 2012, 4:42 PM UTC

JohnHaugeland wrote: post #145

Symmetry wrote: post #116

Earlier quoted context omitted.

Well, maybe. There are a whole lot of very different things called "The Singularity" and some of them are much more reasonable than others. There's the Cambpellian Singularity, which says that we won't be able to predict what will happen next. Pretty non-controversial as far as it goes. There's the Vingean Singularity, which says that if we ever develop AIs that can think as fast and as well as humans then due to Moo…

"There are a whole lot of very different things called "The Singularity" " If the singularity was a legitimate concept with anything approaching experimental evidence, then this could not be true. This observation of yours - with which I agree - suggests to me that The Singularity needs a pope hat. It is instructive to notice that all of "the singularities" are the products of science fiction authors, and in the case…

It certainly lacks experimental evidence, but then again so does all other speculation about the future. That doesn't that it's illegitimate, just that it isn't scientific. On the other hand a lot of the terminology and discussion around "The Singularity" does tend to be confused, and I think that we might all be better off if we stopped using that term.

I should also point out that you're exagerating the link between the idea and science fiction authors. Campbell was a science fiction author (well mostly an editor but close enough) and Vinge was too, though Vinge was also a CS professor. The people I'd associate with the other schools of thought aren't science fiction authors, though.

Re: Building high-level features using large scale unsupervised learning

#155

Post by Dn_Ab » Fri, Jun 22, 2012, 4:43 PM UTC

JohnHaugeland wrote: post #145

Symmetry wrote: post #116

Earlier quoted context omitted.

Well, maybe. There are a whole lot of very different things called "The Singularity" and some of them are much more reasonable than others. There's the Cambpellian Singularity, which says that we won't be able to predict what will happen next. Pretty non-controversial as far as it goes. There's the Vingean Singularity, which says that if we ever develop AIs that can think as fast and as well as humans then due to Moo…

"There are a whole lot of very different things called "The Singularity" " If the singularity was a legitimate concept with anything approaching experimental evidence, then this could not be true. This observation of yours - with which I agree - suggests to me that The Singularity needs a pope hat. It is instructive to notice that all of "the singularities" are the products of science fiction authors, and in the case…

Vinge is a retired math professor, I.J. Good was an accomplished mathematician, Kurzweil makes hard to swallow predictions but is still an accomplished technologist and I happen to very much enjoy Vinge's writing. Regardless, the idea is worth considering independent of who is saying it.

I am also sure you know that words - such as variety and polymorphism - have different context specific meanings. Singularity in this case as in the kind of thing you can find on a variety but not on a manifold.

The idea of infinite recursive Moore's law fueled intelligence explosions leading to super human intellects by 2030 is something I assign a low probability to. I don't find it hard to believe that there is some point in the future - say 2131 - such that if anyone alive today or previously were transported there, they would never be able to understand what was going on and everyone from that time would think circles around them.

Re: Building high-level features using large scale unsupervised learning

#156

Post by Symmetry » Fri, Jun 22, 2012, 4:47 PM UTC

Dn_Ab wrote: post #126

Symmetry wrote: post #116

Earlier quoted context omitted.

Well, maybe. There are a whole lot of very different things called "The Singularity" and some of them are much more reasonable than others. There's the Cambpellian Singularity, which says that we won't be able to predict what will happen next. Pretty non-controversial as far as it goes. There's the Vingean Singularity, which says that if we ever develop AIs that can think as fast and as well as humans then due to Moo…

Your characterization of Vinge's singularity is incorrect. I have never read anything in which he brings up infinity, I.J. Good does though. Vinge's is actually more like your AI revolution except that it will be evident as a singularity only to those looking forward and not to those looking backwards. So instead of using agriculture/industrial divide as an analogy he posits a human/animal divide. As his definition o…

I could have sworn that that argument came from a short piece of non-fiction Vinge wrote which was my first exposure to the whole idea, but I might be mis-remembering because it was a long time ago. Or, given that this was a long time ago, he might have changed his viewpoint.

Re: Building high-level features using large scale unsupervised learning

#157

Post by MaysonL » Fri, Jun 22, 2012, 4:54 PM UTC

JohnHaugeland wrote: post #145

Symmetry wrote: post #116

Earlier quoted context omitted.

Well, maybe. There are a whole lot of very different things called "The Singularity" and some of them are much more reasonable than others. There's the Cambpellian Singularity, which says that we won't be able to predict what will happen next. Pretty non-controversial as far as it goes. There's the Vingean Singularity, which says that if we ever develop AIs that can think as fast and as well as humans then due to Moo…

"There are a whole lot of very different things called "The Singularity" " If the singularity was a legitimate concept with anything approaching experimental evidence, then this could not be true. This observation of yours - with which I agree - suggests to me that The Singularity needs a pope hat. It is instructive to notice that all of "the singularities" are the products of science fiction authors, and in the case…

In general, a singularity is a point at which an equation, surface, etc., blows up or becomes degenerate. Singularities are often also called singular points.

To call Vinge a particularly bad science fiction author says more about your critical acumen than about him. (Perhaps you're thinking about his ex-wife?)

(I read the "singularity is near" in the article title as ironic - almost parodic).

Re: Building high-level features using large scale unsupervised learning

#158

Post by heretohelp » Fri, Jun 22, 2012, 5:22 PM UTC

roel_v wrote: post #56

heretohelp wrote: post #18

Earlier quoted context omitted.

Haha, no. --- last company was in computer vision.

"last company was in computer vision." Did you sell, leave or did it fail? Why? I have some ideas that I think are novel applications of computer vision, and just within the range of what's feasible, but it seems that most computer vision applications look like that at first, and then after 90% done find out that the second 90% is exponentially harder and, realistically, infeasible. How could I test my ideas against…

Company was doing well and had a good idea. Product worked great, we did our job. The problem is, management didn't.

I left after all the other engineers did.

Re: Building high-level features using large scale unsupervised learning

#159

Post by marshallp » Fri, Jun 22, 2012, 5:33 PM UTC

JohnHaugeland wrote: post #135

redmoskito wrote: post #127

Earlier quoted context omitted.

I find both of your comments extremely condescending, both toward saalweachter and the authors of this article. 1. The fact that Claude Shannon succeeded in training a chess system has virtually no impact on sallweachter's claim that many AI results were overstated. 2. Certainly the press overstated them, which supports saalweachter's premise rather than weakening it. Even if the _implied_ claim was that _researchers…

"I find both of your comments extremely condescending" When a comment opens with a tone like this, I usually don't bother to respond, but I'll give you a chance, because you seem to have done a lot of honest mis-reading. To wit, it may be of value for you to inspect your own tone, if you find public condescention inappropriate. . "1. The fact that Claude Shannon succeeded in training a chess system has virtually no i…

I think you guys are getting a little serious.

Also, this paper is about 20,000 object categories, not just 1 (faces). And the neural network is not the standard type but of the deep learning variety which has only existed since 2005 (invented by geoff hinton, who was also big in neural net circles in the 80s so he's not some newcomer who hasn't done his literature search). One of the couthors of the paper is andrew ng, head of the stanford ai lab, so he's pretty legit.

Re: Building high-level features using large scale unsupervised learning

#160

Post by marshallp » Fri, Jun 22, 2012, 5:49 PM UTC

JohnHaugeland wrote: post #142

robrenaud wrote: post #125

Earlier quoted context omitted.

An image is a lot more complicated than a pair of ids and a rating. Counting the number of rows in the training database is misleading. I can build a reasonable dataset for a prediction task from a set of 100M rows from a database that I maintain in my spare time ( http://councilroom.com , predict player actions given partial game states). Don't get me wrong, the Netflix prize was cool. What's cool about this is that…

"An image is a lot more complicated than a pair of ids and a rating." Predicting someone's reaction to a given movie is a lot more complicated than a pair of IDs and a rating, too, it turns out. Let's take the speculation out of this. You can get features of an image with simple large blob detection; four recurring boltzmann machines with half a dozen wires each can find the corners of a nose-bounding trapezoid quite…

The person who invented the boltzman machines - is - the inventor of this technique. He invented boltzman machines in the 80s and spent over 20 years trying to get them to actually work on difficult tasks.

Your rant about this not being compression or whatever you're trying to say is completely off the mark. You don't seem to understand what this work is about.

The netflix challenge is a supervised learning challenge. You have lots of 'labeled data'. This technique is about using 'unlabeled' data.

(Side note: At one point, Geoff Hinton and his group using this technique had the best result in the netflix challenge, but were beaten out by ensembles of algorithms.)

Cyc has nothing to do with this and is huge failure at AI.

tldr; You don't seem to be knowing what you're talking about after having reading your comments, and seem to readily discount the some of the most prominent machine learning researchers in the world today. You're obscuring important results that newcomers might have found interesting to follow up on.