Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrew Ng and the Quest for the New AI

wired.com

Post reply on HN
Print view View original

41–50 of 73 posts

Re: Andrew Ng and the Quest for the New AI

#41

Post by jhartmann »

wookietrader wrote: post #33
jhartmann wrote: post #4

I've said this before, but deep learning is terribly powerful precisely because you don't have to spend lots of time doing feature engineering. Multi layer networks that are trained in semi-supervised, unsupervised and supervised fashions all can now produce networks that meet or beat the state of the art hand created models for speech, handwriting recognition, ocr, and object recognition. We are only just beginning…

This has been said about neural nets two times already. Sadly, they did never deliver. There are still applications where e.g. random forests beat the crap out of all kinds of deep learning algorithms in (a) training time (b) predictive quality (c) prediction time. We should stop hyping this. I am a researcher working in deep learning myself, but the current deep learning hype is actually what makes me worry that I w…

Well I think a big part of this is that right now we have finally gotten to where the algorithms + the required computing power are starting to become more widely available. Cheap graphics cards or things like Intel's Phi + techniques like drop out to prevent overfitting are really enabling much more sophisticated things to be done in a reasonable wall time. Granted multilayer neural networks aren't a free lunch that will solve everything, but there are large classes of problems that are falling to these techniques all the time. We are also finding that Neural Networks scale very well to very large architectures, better than some of the other techniques. I understand we should be careful not to overhype since we have seen previous excitement cause a mass exodus from this research before. I however think people like Hinton were always right, and this was awesome stuff. We just couldn't really take advantage of it because we could never train it for long enough and we hadn't learned how to do things efficiently yet.

Re: Andrew Ng and the Quest for the New AI

#42

Post by ivanist »

pfg wrote: post #38
ivanist wrote: post #31

Earlier quoted context omitted.

Haha me too late to the party. I just joined. Frantically watching the video lectures since the assignments are due today (hard deadline)!

Okay, now you got me scared. The hard deadlines for all my assignments are on July 8th 8:59AM (that's CEST, so it's probably July 7th in PST.)

I took this one https://www.coursera.org/course/ml and it started on Apr 22 and ends around July 1st I think. So not sure what course you are talking about.

Re: Andrew Ng and the Quest for the New AI

#43

Post by DennisP »

why-el wrote: post #37

So I am a little confused. Where are we on the learning part of AI? As I understand it, the current consensus is to throw as much data as you can at your model (millions of cat pictures in this article's example) to make it pick up patterns and yet still claim that we are closing in on how the brain works? As far as I can tell no human brain would need that many pictures to see a pattern. In fact, and this is probabl…

Bear in mind that the neural network is starting at the same point as a newborn baby.

Re: Andrew Ng and the Quest for the New AI

#44

Post by piglop »

why-el wrote: post #37

So I am a little confused. Where are we on the learning part of AI? As I understand it, the current consensus is to throw as much data as you can at your model (millions of cat pictures in this article's example) to make it pick up patterns and yet still claim that we are closing in on how the brain works? As far as I can tell no human brain would need that many pictures to see a pattern. In fact, and this is probabl…

You may not have seen millions of cats, but how many images have your brain processed since you can see? A five years old brain has been trained on billions of images (along with many other simultaneous inputs).

Re: Andrew Ng and the Quest for the New AI

#45

Post by pfg »

ivanist wrote: post #42
pfg wrote: post #38

Earlier quoted context omitted.

Okay, now you got me scared. The hard deadlines for all my assignments are on July 8th 8:59AM (that's CEST, so it's probably July 7th in PST.)

I took this one https://www.coursera.org/course/ml and it started on Apr 22 and ends around July 1st I think. So not sure what course you are talking about.

Well, this is weird. I'm taking the exact same course and here's what I see for the first programming assignment: http://i.imgur.com/GyEwcUG.png (same with review questions)

Re: Andrew Ng and the Quest for the New AI

#46

Post by Lambdanaut »

lomendil wrote: post #36
kinofcain wrote: post #2

There is a very quick reference to the person who inspired him, Jeff Hawkins, whose book is worth a read: http://www.amazon.com/On-Intelligence-Jeff-Hawkins/dp/080507... Edit: update link

I just saw Jeff Hawkins give a talk and it was quite interesting. I was a bit worried, however, that he is basing his theory of intelligence on the human neocortex, while claiming to go after general principles. This is guaranteed not to be terribly general, considering the many bits of matter on this planet that exhibit intelligence without a neocortex. By many, I mean ones that hugely outnumber humans. So very inte…

In "On Intelligence" he postulates that he's not necessarily going for a "human-like" intelligence or even a "life-like" one.

Basically he just wants something that's very good at recognizing patterns over time, which I can imagine the neocortex would be great at.

Though, he also references the thalamus and hippocampus in the books a lot, as very important parts of the brain to his framework. [http://en.wikipedia.org/wiki/Memory-prediction_framework#Neu...]

Re: Andrew Ng and the Quest for the New AI

#47

Post by wookietrader »

jhartmann wrote: post #41
wookietrader wrote: post #33

Earlier quoted context omitted.

This has been said about neural nets two times already. Sadly, they did never deliver. There are still applications where e.g. random forests beat the crap out of all kinds of deep learning algorithms in (a) training time (b) predictive quality (c) prediction time. We should stop hyping this. I am a researcher working in deep learning myself, but the current deep learning hype is actually what makes me worry that I w…

Well I think a big part of this is that right now we have finally gotten to where the algorithms + the required computing power are starting to become more widely available. Cheap graphics cards or things like Intel's Phi + techniques like drop out to prevent overfitting are really enabling much more sophisticated things to be done in a reasonable wall time. Granted multilayer neural networks aren't a free lunch that…

Yes you are right.

Still, deep learning has done nothing more than classification right now.

What about predictive distributions, regression of complicated outputs (e.g. periodic data) and, most of all, heterogenous inputs? Right: nothing impressive has been done in that area, despite of huge amounts of practical problems.

Let's see if deep learning generalizes to those things. If it does (and I personally believe so) let's be happy. Before that, we still have to envy what Gaussian processes, Gradient boosting machines and random forests can do what DL so far cannot.

Re: Andrew Ng and the Quest for the New AI

#48

Post by SCdF »

drcross wrote: post #9
aespinoza wrote: post #6

I thought the man behind the Google Brain was Ray Kurzweil ( http://www.wired.com/business/2013/04/kurzweil-google-ai/ ).

Kurzweil is a hack. (Source: I've an Msc in Robotics and read all his books)

drcross is a hack. (Source: I'm a space fairy)

If you're going to throw broad statements around some examples would really help lend you credibility.

Re: Andrew Ng and the Quest for the New AI

#49

Post by cscurmudgeon »

Ah. The good old AI cycle.

Scientist: X can help us get full AI!

You: Why?

Scientist: Because of reason R.

You: But, reason R is a non sequitur...

More seriously, reasons similar to that for deep learning have been repeated multiple times in AI with failure (e.g. Thinking Machines).

I would suggest that these folks remain calm and build something on the scale of IBM's Watson using just deep learning..

Re: Andrew Ng and the Quest for the New AI

#50

Post by imurray »

wookietrader wrote: post #47
jhartmann wrote: post #41

Earlier quoted context omitted.

Well I think a big part of this is that right now we have finally gotten to where the algorithms + the required computing power are starting to become more widely available. Cheap graphics cards or things like Intel's Phi + techniques like drop out to prevent overfitting are really enabling much more sophisticated things to be done in a reasonable wall time. Granted multilayer neural networks aren't a free lunch that…

Yes you are right. Still, deep learning has done nothing more than classification right now. What about predictive distributions, regression of complicated outputs (e.g. periodic data) and, most of all, heterogenous inputs? Right: nothing impressive has been done in that area, despite of huge amounts of practical problems. Let's see if deep learning generalizes to those things. If it does (and I personally believe so…

Still, deep learning has done nothing more than classification right now. What about predictive distributions, regression of complicated outputs...

http://homepages.inf.ed.ac.uk/imurray2/pub/12deepai/ has predictive distributions from deep learning, passed on to time-series smoothing for articulatory inversion. It's a previous neural net approach made deep, and working better as a result.

(I agree that like any machine learning framework, neural networks have their strengths and weaknesses, and open challenges.)

Post reply on HN