Improving the way neural networks learn
neuralnetworksanddeeplearning.com
Improving the way neural networks learn
1–10 of 20 posts
Re: Improving the way neural networks learn
#2>You have to realize that our theoretical tools are very weak. Sometimes, we have good mathematical intuitions for why a particular technique should work. Sometimes our intuition ends up being wrong [...] The questions become: how well does my method work on this particular problem, and how large is the set of problems on which it works well.
I'm not very familiar with this field. Has anyone made any progress on formalizing ways to measure the capabilities of intelligent systems? If the theory is weak, there must be someone working on improving it, right?
Re: Improving the way neural networks learn
#3I found that the statement about the cross entropy not true. When y==a the function is non-monotonic with 0 at the extremes but not at the middle. So the "proof" shown is confusing to me.
Re: Improving the way neural networks learn
#4>You have to realize that our theoretical tools are very weak. Sometimes, we have good mathematical intuitions for why a particular technique should work. Sometimes our intuition ends up being wrong [...] The questions become: how well does my method work on this particular problem, and how large is the set of problems on which it works well. I'm not very familiar with this field. Has anyone made any progress on form…
Probably Dileep George et al. at Vicarious....
But since that's a $55M Black Hole with no published results other than a mostly meaningless claim to having solved Captcha (which wasn't all that tough a task to begin with), there's no way to tell since it doesn't seem like practitioners of the art are the ones evaluating his prospects for further funding. But don't believe some random dude on HN, here's Yann Le Cun saying pretty much the same thing:
Re: Improving the way neural networks learn
#5I found that the statement about the cross entropy not true. When y==a the function is non-monotonic with 0 at the extremes but not at the middle. So the "proof" shown is confusing to me.
Which statement about cross entropy is confusing?
Re: Improving the way neural networks learn
#6>You have to realize that our theoretical tools are very weak. Sometimes, we have good mathematical intuitions for why a particular technique should work. Sometimes our intuition ends up being wrong [...] The questions become: how well does my method work on this particular problem, and how large is the set of problems on which it works well. I'm not very familiar with this field. Has anyone made any progress on form…
Probably Dileep George et al. at Vicarious.... But since that's a $55M Black Hole with no published results other than a mostly meaningless claim to having solved Captcha (which wasn't all that tough a task to begin with), there's no way to tell since it doesn't seem like practitioners of the art are the ones evaluating his prospects for further funding. But don't believe some random dude on HN, here's Yann Le Cun sa…
The OP is quoting LeCun, so this is not a coincidence!
Re: Improving the way neural networks learn
#7Earlier quoted context omitted.
Probably Dileep George et al. at Vicarious.... But since that's a $55M Black Hole with no published results other than a mostly meaningless claim to having solved Captcha (which wasn't all that tough a task to begin with), there's no way to tell since it doesn't seem like practitioners of the art are the ones evaluating his prospects for further funding. But don't believe some random dude on HN, here's Yann Le Cun sa…
The OP is quoting LeCun, so this is not a coincidence!
Hey Michael, I loved your book on Quantum Computing, but don't get me started on D-Wave or as I see it: $15M for a huge magic box that might be faster than a $15,000 GPU cluster for some problems.
But seriously, the book rocked, and this one's coming along nicely.
Re: Improving the way neural networks learn
#8The sections on Regularization and Dropout have some amazing prose. I haven't read any of the other chapters, but just skimming through those sections have helped enlighten me on quite a few things that have confused me for years in completely different mediums...Such as why a random forest made up of randomly selected simple CARTs generally predicts better than a single complex CART, or why fitting a distribution to empirical data can benefit from using AIC or BIC methods.
Re: Improving the way neural networks learn
#9Neural Networks class on coursera covered a lot of the same topics with both heavy math theory crafting and hefty amount of practical application. https://www.coursera.org/course/neuralnets
Re: Improving the way neural networks learn
#10Neural Networks class on coursera covered a lot of the same topics with both heavy math theory crafting and hefty amount of practical application. https://www.coursera.org/course/neuralnets
It's a pity the lectures are not accessible any longer. How i wish i could find them somewhere!