What does a neural network actually do?
moalquraishi.wordpress.com
What does a neural network actually do?
1–10 of 39 posts
Re: What does a neural network actually do?
#2They write comments on HN.
Re: What does a neural network actually do?
#3I've found that, in practice, traditional neural networks tend to be prone to overfitting and are finicky about their parameters (in particular the topology and number of nodes you choose).
I use the word "traditional" to describe the NN architecture discussed in the article. Recent NN research has been promising [1], but this article strictly discusses traditional NN's. I don't really have much experience with the newer NN algorithms, so I'm not sure to what extent they suffer from the same problems as traditional NN's.
[1] http://en.wikipedia.org/wiki/Neural_network#Recent_improveme...
Re: What does a neural network actually do?
#4Looks like an interesting text, but to be honest I didn't understand a substantial portion of it.
Re: What does a neural network actually do?
#5They write comments on HN.
Those are Markov-chain based text generators. (yes, I got the joke).
Re: What does a neural network actually do?
#6I've found that, in practice, traditional neural networks tend to be prone to overfitting and are finicky about their parameters (in particular the topology and number of nodes you choose). I use the word "traditional" to describe the NN architecture discussed in the article. Recent NN research has been promising [1], but this article strictly discusses traditional NN's. I don't really have much experience with the n…
Is it safe to say that in ML, use of NNs is more about writing code that designs NNs, evaluates the results, and modifies the designs to optimize some desired meta-values, like accuracy, efficiency, etc?
Re: What does a neural network actually do?
#7Looks like an interesting text, but to be honest I didn't understand a substantial portion of it.
You're 17; get to work understanding it! The more you learn now, the more you'll be able to do for the rest of your life. Plus, diving into random topics and understanding them more deeply than anyone else is a ton of fun.
Re: What does a neural network actually do?
#8"What does a neural network actually do?"
This is a fundamental Question: Can we really say and predict what a neural network does?
Contrary to an engineered / constructed algorithm a neural network is 'trained'.
Whenever we will present a 'known' input pattern it will reposnd with a 'learned' response.
This however introduces interesting problems: How can we _debug_ a neural network? How can we debug a correlation? Sure we can tune its paramters, we can train it some more to again show the desired response. But reaching this point we just abandoned knowing how the intrinsic algorithm works in favor to just focussing on the result.
Okay - now if we follow this argument - this would lead to: If we simulate the whole brain by simulating the neural network, we won't gain any knowlege about the intrinsic workings of the brain. We won't find any enlightenment about the innermost algorithm _represented_ by the neural network we call our brain.
Re: What does a neural network actually do?
#9I've found that, in practice, traditional neural networks tend to be prone to overfitting and are finicky about their parameters (in particular the topology and number of nodes you choose). I use the word "traditional" to describe the NN architecture discussed in the article. Recent NN research has been promising [1], but this article strictly discusses traditional NN's. I don't really have much experience with the n…
Hinton's DropOut [1] and Wan's DropConnect [2] have ameliorated some of the overfitting issues present in traditional NN's. In fact, DropConnect in conjunction with deep learning are responsible for new records being set on classical datasets such as MNIST.
[1] http://arxiv.org/pdf/1207.0580.pdf [2] http://cs.nyu.edu/~wanli/dropc/
Re: What does a neural network actually do?
#10They write comments on HN.
Those are Markov-chain based text generators. (yes, I got the joke).
Well recently a lot of work has gone into using NNs for natural language processing. Typically it's trained to do something like predict the next word or character in a sequence. Using that you could possibly create a far better generative model than markov chains, and create more realistic sentences. Perhaps you could even combine them (NN gets the output of the markov chain to help make it's prediction.)