Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Andrew Ng and the Quest for the New AI

wired.com

Post reply on HN
Print view View original

61–70 of 73 posts

Re: Andrew Ng and the Quest for the New AI

#61

Post by pseut »

Cushman wrote: post #60
pseut wrote: post #58

Earlier quoted context omitted.

No, I think the purpose of citing claims is to demonstrate that they have validity. People can disagree about what constitutes "validity" and the citation is often inappropriate. But you shouldn't intentionally cite a source that shows your claim has no support, you just shouldn't make the claim. (at least, that's my impression of hnetiquette)

Academic citation isn't an honor system . They're there so you can look them up. That said, your point is effectively what I was saying in that the statement "I have personal experience with this" taken in good faith is much more supportive than a link which does not actually support my point, yet the latter frequently passes without comment.

They're there so that you can look them up, but they're expected to justify your claim. It's not a scavenger hunt either.

Re: Andrew Ng and the Quest for the New AI

#62

Post by niklaslogren »

Very interesting article, it makes me hopeful.

This might be slightly off-topic, but I'll try it here anyway: can anyone recommend any books/other learning resources for someone who wants to grasp neural networks?

I'm a CS student who finds the idea behind them really exciting, but I'm not sure where to get started.

Re: Andrew Ng and the Quest for the New AI

#63

Post by jcrites »

jbelanich wrote: post #27
gavanwoolery wrote: post #13

Hmm...I don't mean to be a skeptic, but I do not see any new theories here. Neural networking has been around for a long time, as have an abundance of theories and implementations around it...some people have gone so far as to build an actual brain replica (a digital version of the bio/analog thing). Neural networking is extremely powerful, but to be of any use, you need a lot of computing power. As it turns out, our…

You're correct in that neural networks as a model have been around for a long time. However, those networks were restricted to be shallow because backpropogation didn't work well on networks with many hidden layers. Only recently have researchers developed learning procedures that can learn these deep architectures efficiently, using some clever unsupervised learning techniques. And surprisingly, they are finding tha…

> Only recently have researchers developed learning procedures that can learn these deep architectures efficiently, using some clever unsupervised learning techniques.

Would you mind naming some of these techniques, if you're familiar with them? I'd like to take a deeper look.

Re: Andrew Ng and the Quest for the New AI

#64

Post by visarga »

wookietrader wrote: post #33
jhartmann wrote: post #4

I've said this before, but deep learning is terribly powerful precisely because you don't have to spend lots of time doing feature engineering. Multi layer networks that are trained in semi-supervised, unsupervised and supervised fashions all can now produce networks that meet or beat the state of the art hand created models for speech, handwriting recognition, ocr, and object recognition. We are only just beginning…

This has been said about neural nets two times already. Sadly, they did never deliver. There are still applications where e.g. random forests beat the crap out of all kinds of deep learning algorithms in (a) training time (b) predictive quality (c) prediction time. We should stop hyping this. I am a researcher working in deep learning myself, but the current deep learning hype is actually what makes me worry that I w…

It's a little offtopic, but can you recommend a random forest implementation to be used from the command line, like Vowpal Wabbit does for linear learning? I like VW for its speed and ease of use.

Re: Andrew Ng and the Quest for the New AI

#65

Post by visarga »

mark_l_watson wrote: post #26
hvs wrote: post #5

OT: His online Machine Learning class last year was great. He is the best professor I've ever had, and explains things so clearly that you understand them the first time. You are lucky if you ever get to work or study under him.

I second this! I have taken 7 Coursera classes, and most of them "lightly": just doing as much work as I needed to for passing the class, with just a few classes that I did put a lot of energy into. Andrew's class was in this second category: I kept taking the tests and tweaking the homework assignments over and over again until I got a 99.5% score in the class. His class was lots of fun and also very useful material…

Yes, unfortunately his partner, Daphne Koller, though very esteemed academically, made me feel very frustrated in the PGM class. Not the same great feeling of clarity.

Re: Andrew Ng and the Quest for the New AI

#66

Post by visarga »

pfg wrote: post #25
psbp wrote: post #11

Earlier quoted context omitted.

I wish Coursera followed the Udacity model. I always find out about these classes after they're already weeks in progress or over.

You can star any Coursera class to receive notifications whenever new sessions are announced. Also, I believe it's still possible to join the current session (first assignment was due this weekend, but you can turn it in late with just a 20% penalty.)

I take new courses at any time, even if they have ended. Later on, when they recycle, I can do them all over again with ease.

Re: Andrew Ng and the Quest for the New AI

#67

Post by visarga »

Jach wrote: post #54
EthanHeilman wrote: post #23

We should stop trying to claim every new method is "like the brain". We don't have any clear understanding of how the brain works. One can be inspired by a particular and likely wrong cognitive theory, but one can not say one is building "machines that can process data in much the same way the brain does" truthfully without a deeper, and currently unavailable, understanding of the functioning of the human brain.

I feel the same way, that researchers should stop trying to mimic the brain, but not because we don't understand the brain. While I think there are still several decades before we'll be able to have mind uploads, I also think a lot of people underestimate the quality of modern brain science. In any case, I have the same reason as Dijkstra for why I think mimicking the brain isn't that great an idea. In http://www.cs.…

They are trying to find THE ONE ALGORITHM that solves all A.I. problems. The brain has an implementation of it, but it is in wetware, hard to extract. Deep learning makes some pretty good approximations of the visual areas, though.

Re: Andrew Ng and the Quest for the New AI

#68

Post by visarga »

jcrites wrote: post #63
jbelanich wrote: post #27

Earlier quoted context omitted.

You're correct in that neural networks as a model have been around for a long time. However, those networks were restricted to be shallow because backpropogation didn't work well on networks with many hidden layers. Only recently have researchers developed learning procedures that can learn these deep architectures efficiently, using some clever unsupervised learning techniques. And surprisingly, they are finding tha…

> Only recently have researchers developed learning procedures that can learn these deep architectures efficiently, using some clever unsupervised learning techniques. Would you mind naming some of these techniques, if you're familiar with them? I'd like to take a deeper look.

Restricted Boltzmann Machines. https://www.youtube.com/watch?v=AyzOUbkUf3M

This video drives the point home, and is made by the author of this technique.

Re: Andrew Ng and the Quest for the New AI

#69

Post by wookietrader »

imurray wrote: post #50
wookietrader wrote: post #47

Earlier quoted context omitted.

Yes you are right. Still, deep learning has done nothing more than classification right now. What about predictive distributions, regression of complicated outputs (e.g. periodic data) and, most of all, heterogenous inputs? Right: nothing impressive has been done in that area, despite of huge amounts of practical problems. Let's see if deep learning generalizes to those things. If it does (and I personally believe so…

Still, deep learning has done nothing more than classification right now. What about predictive distributions, regression of complicated outputs... http://homepages.inf.ed.ac.uk/imurray2/pub/12deepai/ has predictive distributions from deep learning, passed on to time-series smoothing for articulatory inversion. It's a previous neural net approach made deep, and working better as a result. (I agree that like any machi…

Okay, I should have worded that differently. There is also a paper of Salakhutdinov learning a kernel for Gaussian processes. That'd account for that as well.

My point is (I did not really write that above) that deep learning does not stand unchallenged in this domain. Its dominance is so far "only" apparent in vision and audio classification tasks.

Re: Andrew Ng and the Quest for the New AI

#70

Post by pilooch »

It seems the relationship between the brain and deep learning has evolved in such a way that the later can help with insights into how the former works.

In this regard, I thought I would mention the extraordinary simple and elegant talk by G. Hinton last summer: http://www.youtube.com/watch?v=DleXA5ADG78

It starts from a simple and clever improvement to an existing deep learning method and ends up with beautiful (and simple!) insights on why neurons are using simple spikes to communicate.

Post reply on HN