Earlier quoted context omitted.
I'm not a Machine Learning / AI expert, so I have to ask: if running a neural network on 16,000 cores with a training set of 10 million objects isn't cutting edge research -- and if running "far larger" networks than this, as you say you have, also isn't cutting edge research -- then please tell me: what is cutting edge research? I ask this question in all seriousness; I'd really like to know. (And yes, I see that yo…
I'm not a large scale ML person, and not intending to take away from the achievement of the team in the OP, but experiments in large scale, unsupervised learning have been going for a long time (even using the autoencoder approach). When you think about it, large scale requires unsupervised... Here is an old example with hundreds of millions of records and instances: http://aaaipress.org/Papers/KDD/1998/KDD98-028.pdf…
Building high-level features using large scale unsupervised learning
161–170 of 196 posts
Re: Building high-level features using large scale unsupervised learning
#162Earlier quoted context omitted.
"An image is a lot more complicated than a pair of ids and a rating." Predicting someone's reaction to a given movie is a lot more complicated than a pair of IDs and a rating, too, it turns out. Let's take the speculation out of this. You can get features of an image with simple large blob detection; four recurring boltzmann machines with half a dozen wires each can find the corners of a nose-bounding trapezoid quite…
The person who invented the boltzman machines - is - the inventor of this technique. He invented boltzman machines in the 80s and spent over 20 years trying to get them to actually work on difficult tasks. Your rant about this not being compression or whatever you're trying to say is completely off the mark. You don't seem to understand what this work is about. The netflix challenge is a supervised learning challenge…
Your reading skills seem to be up to par, since I have discredited a list of zero people.
"You're obscuring important results that newcomers might have found interesting to follow up on."
Not only have I obscured no results, but this isn't actually something I have the power to do.
Re: Building high-level features using large scale unsupervised learning
#163Earlier quoted context omitted.
I'm not a Machine Learning / AI expert, so I have to ask: if running a neural network on 16,000 cores with a training set of 10 million objects isn't cutting edge research -- and if running "far larger" networks than this, as you say you have, also isn't cutting edge research -- then please tell me: what is cutting edge research? I ask this question in all seriousness; I'd really like to know. (And yes, I see that yo…
I'm not a large scale ML person, and not intending to take away from the achievement of the team in the OP, but experiments in large scale, unsupervised learning have been going for a long time (even using the autoencoder approach). When you think about it, large scale requires unsupervised... Here is an old example with hundreds of millions of records and instances: http://aaaipress.org/Papers/KDD/1998/KDD98-028.pdf…
Not necessarily. Crowdsourcing is another option, like Google's image tagging game, reCAPTCHA, et cetera.
Pay a herd of people to do things, and they'll do things for you. You don't have to pay them in money. Telling them they have a high score is often enough.
Re: Building high-level features using large scale unsupervised learning
#164Earlier quoted context omitted.
"There are a whole lot of very different things called "The Singularity" " If the singularity was a legitimate concept with anything approaching experimental evidence, then this could not be true. This observation of yours - with which I agree - suggests to me that The Singularity needs a pope hat. It is instructive to notice that all of "the singularities" are the products of science fiction authors, and in the case…
Vinge is a retired math professor, I.J. Good was an accomplished mathematician, Kurzweil makes hard to swallow predictions but is still an accomplished technologist and I happen to very much enjoy Vinge's writing. Regardless, the idea is worth considering independent of who is saying it. I am also sure you know that words - such as variety and polymorphism - have different context specific meanings. Singularity in th…
What I was getting at was "you realize they're writing books to make people happy for money, not doing legitimate science on that day, right?"
.
"words - such as variety and polymorphism - have different context specific meanings."
Sure. All handwaving about the rules of language notwithstanding, though, none of The Singularities have merit or underlying measurement, even if you want to talk syntax and grammar to create a seeming of academia by proxy.
.
"The idea of infinite recursive Moore's law"
... is nonsense. What would "recursion" be in the context of Moore's law? Have you even thought this over?
What, Moore's Law solves itself by going deeper into itself until the datastructure is exhausted?
.
"fueled intelligence explosions"
The science fiction part. I mean, you might as well say "fuelled by warp drives," because there's no evidence they're going to happen either. Or unicorns.
.
"is something I assign a low probability to."
This suggests that you don't know what probabilities are. Probabilities are either frequentist, which cannot happen here because we have no knowledge of the rates here (this would be like calculating the frequentist probability of alien life - it's just making numbers up,) or Bayesian, where you draw probabilities from observed events, at which point the probability is exactly zero.
So, is it undefined or zero that you're promoting?
.
"I don't find it hard to believe that there is some point in the future - say 2131"
(rolls eyes)
.
"they would never be able to understand what was going on and everyone from that time would think circles around them."
It seems you don't even need to be transported into the future for that.
Re: Building high-level features using large scale unsupervised learning
#165Earlier quoted context omitted.
"There are a whole lot of very different things called "The Singularity" " If the singularity was a legitimate concept with anything approaching experimental evidence, then this could not be true. This observation of yours - with which I agree - suggests to me that The Singularity needs a pope hat. It is instructive to notice that all of "the singularities" are the products of science fiction authors, and in the case…
In general, a singularity is a point at which an equation, surface, etc., blows up or becomes degenerate. Singularities are often also called singular points. To call Vinge a particularly bad science fiction author says more about your critical acumen than about him. (Perhaps you're thinking about his ex-wife?) (I read the "singularity is near" in the article title as ironic - almost parodic).
Do you also participate in discussions of Microsoft Surface by explaining that in general, a surface is a flat exterior of a coherent object?
.
"To call Vinge a particularly bad science fiction author"
Vinge didn't come up the singularity; Kurtzweil did. Kurtzweil did. I quite like Vernor Vinge's work.
.
"says more about your critical acumen"
Acumen is the ability to make good numeric estimations on the spot, such as business decisions.
Despite that I quite enjoy Vernor Vinge's work, I also feel it important to point out that merely should someone dislike an author you like would not, in fact, be a measurable sleight against their intelligence, any more than liking different pizza toppings would be.
.
"I read the "singularity is near" in the article title as ironic"
That's interesting. If that's correct, then you have a point. (Also, bravo for being part of the one percent of the internet who knows what that word correctly means. I mean that in earnest.)
Re: Building high-level features using large scale unsupervised learning
#166Earlier quoted context omitted.
"There are a whole lot of very different things called "The Singularity" " If the singularity was a legitimate concept with anything approaching experimental evidence, then this could not be true. This observation of yours - with which I agree - suggests to me that The Singularity needs a pope hat. It is instructive to notice that all of "the singularities" are the products of science fiction authors, and in the case…
It certainly lacks experimental evidence, but then again so does all other speculation about the future. That doesn't that it's illegitimate, just that it isn't scientific. On the other hand a lot of the terminology and discussion around "The Singularity" does tend to be confused, and I think that we might all be better off if we stopped using that term. I should also point out that you're exagerating the link betwee…
We disagree here. The singularity made measurable predictions, and not only has every single one that's come to pass failed without exception, but the remainder are no closer to happening than the day they were made.
Remember, originally, computers were supposed to be smarter than us back in the 90s, when people still thought The Simpsons was funny.
.
"I think that we might all be better off if we stopped using that term."
We agree, though I think for different reasons. If I understand you correctly, you are suggesting that we forego this term in favor of clearer, better defined ones, but keep the idea.
I think we should actually reject the concept.
.
"Campbell was a science fiction author (well mostly an editor but close enough) and Vinge was too, though Vinge was also a CS professor. The people I'd associate with the other schools of thought aren't science fiction authors, though."
Minsky, who associates with that school of thought, is. Kurtzweil, who originated this school of thought, is. Stanislaw Ulam, Damien Broderick, Hans Moravec, Greg Egan, Nancy Kress, Larry Niven, Dean Ing, Samuel Delaney, Ray Solomonoff, Pohl, Aasimov, Steele, other-Steele, Yudkowsky, the founder of Singularity University (which is not legally a university) Peter Diamandis, Aubrey de Gray, et cetera.
Sort of the germane understanding is to look at their work. Whose work do they all point to? Moore, Lanier, Holland, and Hawkins.
Guess what all four of them say they think?
.
"I should also point out that you're exagerating the link between the idea and science fiction authors."
I don't think that I am. Every proponent of The Singularity I am aware of writes speculative fiction for money, without exception. The list I gave above isn't even close to exhaustive.
And, again, that's not the actual point I'm making. This isn't about "the link between" The Various Singularities and speculative fiction; I'm asserting directly that every single Singular proposal is itself science fiction.
I'm not saying it's written by science fiction people. Aasimov did legitimate speculative engineering, for example, in the cases of geosynchronous orbits and the space elevator, and arguably in arcological discussion. I would not say that his work was science fiction, despite that he's a science fiction author.
Why?
Because there are real numbers involved. There's real math. There are real equations. He knew the fuel demands, the weight of the building, the energy requirements. He wasn't writing fun stories; he was doing real work.
Spend two weeks. You will _never_ find real work done around the singularity. It hasn't been done.
Earlier you suggested that all speculation about the future was so; I do not agree. We have several quite good designs for an arcology on Mars which would actually work. Seward's Folly is speculative, but it's actual legitimate work; it could be built.
The Singularity is just a cute story.
.
"The people I'd associate with the other schools of thought aren't science fiction authors, though."
Schools of thought is a great phrase to try to bring gravitas to a situation where it isn't warranted.
I'll pay more attention when I see a single work on The Singularity which could pass muster as a freshman thesis at a second string state school.
There's lots of material out there about infinite energy devices, too. Ask yourself a question: if you didn't have the Second Law of Thermodynamics, how would you tell the free energy devices apart from the legitimate ones?
How is it that you know Andrea Rossi doesn't really have cold fusion?
Try some skepticism. It's delicious, and low cholesterol.
Re: Building high-level features using large scale unsupervised learning
#167Earlier quoted context omitted.
I'm not trolling, I've read Keurzweil's book... It's extrapolating. If I was a risk assesor, I'd err on the side of a mass extinction event over us consuming the entire universe as data.
There's already been 4 extinction events, it's pretty safe to bet that it will happen again. But they happen 50-100 million years between each other and usually take thousands of years to take full effect once they begin. Even if technological singularity takes an extra 100-200 years to really happen, if any significant 'AI' is achieved, a lot could happen in a thousand years, let alone a million.
Re: Building high-level features using large scale unsupervised learning
#168Earlier quoted context omitted.
That's what I did a few months ago - quit my job and decided to go to a grad school to study AI (with focus on neural nets and ML).
What did you learn so far about neural nets? I recently looked into machine learning and naively thought I could find at least one practical fun tutorial "Here is a neural network API in C, you have to do that and this to let a simulated robot evade obstacles or learn to play Asteroids". Instead my (extremely superficial) search did find that neural nets are pretty arcane, genetic algorithms trapped in local minima a…
Yes, for some problems it might be faster and better to code logic yourself, but there are also tasks (such as pattern recognition) where NNs might be more effective.
Re: Building high-level features using large scale unsupervised learning
#169Earlier quoted context omitted.
Vinge is a retired math professor, I.J. Good was an accomplished mathematician, Kurzweil makes hard to swallow predictions but is still an accomplished technologist and I happen to very much enjoy Vinge's writing. Regardless, the idea is worth considering independent of who is saying it. I am also sure you know that words - such as variety and polymorphism - have different context specific meanings. Singularity in th…
Maybe you didn't realize this, but the science fiction author bit wasn't meant as a slander. Many high quality people, and also Kurtzweil, are science fiction authors. What I was getting at was "you realize they're writing books to make people happy for money, not doing legitimate science on that day, right?" . "words - such as variety and polymorphism - have different context specific meanings." Sure. All handwaving…
2.) Singularity as in breakdown not as in single. You purposely muddled the meaning to make your quip work.
3.) Moore's law fueled as in AI gets interest on their intelligence. Recursive as in AI makes smarter AI makes smarter AI...
4.) Science fiction or not i find it unlikely.
5.) Bayesian. Look up prior.
6.) 2131 was tongue in cheek.
7.) Thanks ;) You actually never address my main point though.
Re: Building high-level features using large scale unsupervised learning
#170I'm seriously considering quitting my job and studying ML for a few months in a desperate attempt to get work in projects like this. I feel like I'm missing out but too dumb for traditional grad school.
Have you seen the online courses? https://www.coursera.org/course/ml (From one of the authors of this paper!) https://www.coursera.org/course/vision https://www.coursera.org/course/computervision Prof. Hinton's videos are very watchable: http://www.youtube.com/watch?v=AyzOUbkUf3M http://www.youtube.com/watch?v=VdIURAu1-aU