Can somebody explain what I'm watching when I press play?
The dots are the training data. The orange and blue background are the estimation for how the neural net will classify a new orange or blue dot.
Tinker with a Neural Network in Your Browser
31–40 of 122 posts
Re: Tinker with a Neural Network in Your Browser
#32In case you are an idiot like me, you have to train your neural network by pressing "play".
"Okay, I don't understand. Why is my output so terrible?"
I saw the play button very clearly when the page loaded, then promptly got distracted by all the dials and knobs. :-P
Re: Tinker with a Neural Network in Your Browser
#33Earlier quoted context omitted.
Hopefully this helps (correct me if I'm wrong, I'm still learning about neural nets): Think of the whole neural net as a function: input * weight = output At each iteration, we feed in the input to the neural net. Then the neural net compares what output it gets to the correct output. For example, input1 is 5, and the correct output for input1 should have been 2. But the neural net got 3 as the output. So it then dec…
Okay, so it works by minimizing (equiv. maximizing) some function. But that doesn't say much about how it "learns" the gradient. What function does it care about? Average squared error (predict_prob-Z_i)^2 ? Average absolute error? The likelihood function of some assumed distribution? Maximum distance between the classification border and closest observed points? If I saw someone carrying a bag full of blueberries an…
It depends on what you are trying to achieve. There are many, for e.g. see: http://cs231n.github.io/neural-networks-2/#losses
Re: Tinker with a Neural Network in Your Browser
#34This demonstration goes really well with Michael Nielsen's http://neuralnetworksanddeeplearning.com/. At the bottom of the page the author gives a shout out to Nielsen, Bengio, and others.
For someone (like me) who's done a bit of reading but not much implementation, this playground is fantastic!
Re: Tinker with a Neural Network in Your Browser
#35Neat stuff, fun to play with. I wasn't able to get a net to classify the swiss roll. Last time I was playing around with this stuff I found the single biggest factor in the success was the optimizer used. Is this just using a simple gradient descent? I would like to see a drop down for different optimizers.
http://imgur.com/ypBQEWx Add some noise, and use all the inputs, and one 8 wide hidden layer edit: works better with a sigmoid activation curve, but it converges more slowly
Using syn, cos, x1, x2 with 1 six-neuron hidden layer does the trick quickly:
http://imgur.com/UMv5gsH
No need to mess with noise or regularization :)
Re: Tinker with a Neural Network in Your Browser
#36Earlier quoted context omitted.
Yeah I feel like we need some decent understanding of neural networks to have more context on this. Its kind of like being given a specialized shovel but not knowing why you need it or why you should dig holes.
I think it's a playground in the best sense of the term. Take some time and actually play with it, and a lot of fun stuff happens, lightbulbs go off, etc. If you're expecting a lesson, you'll likely be disappointed, but I think there's real value in a true playground. I think the biggest improvement would be if, when hovering over a 'neuron', you get a visual representation of what feeds into it.
For me in Chrome on OSX you do get a visual representation of the neuron's input when hovering. It shows up behind the data points in place of the neurons' output when hovering.
Re: Tinker with a Neural Network in Your Browser
#37Can somebody explain what I'm watching when I press play?
Hopefully this helps (correct me if I'm wrong, I'm still learning about neural nets): Think of the whole neural net as a function: input * weight = output At each iteration, we feed in the input to the neural net. Then the neural net compares what output it gets to the correct output. For example, input1 is 5, and the correct output for input1 should have been 2. But the neural net got 3 as the output. So it then dec…
If I didn't get this course[1], I wouldn't understand what you are talking about.
Re: Tinker with a Neural Network in Your Browser
#38Can somebody explain what I'm watching when I press play?
Super basic explanation:
It's training a neural network to classify a data set with two classes (orange or blue) and the data has two features (x1 or x2). All the orange and blue dots are the training data. So if you take a dot on the graph with coordinates (-2, 4) and it's blue, that would mean that a data point with x1 = -2 and x2 = 4 has the class blue.
You can think of a neural network as a function that can take in arbitrary features (in this case x1 and x2) and tries to output the correct class. That's what the orange and blue colors in the background are, the neural network's guess at the correct classification for any given point (x1, x2).
When you hit play, it iterates through the training data making adjustments to each neuron in the network so that it gets closer to predicting the right class.
If you want to see how well the neural network performs on data it wasn't trained on, you can click "show test data".
Re: Tinker with a Neural Network in Your Browser
#39Neat stuff, fun to play with. I wasn't able to get a net to classify the swiss roll. Last time I was playing around with this stuff I found the single biggest factor in the success was the optimizer used. Is this just using a simple gradient descent? I would like to see a drop down for different optimizers.
http://imgur.com/ypBQEWx Add some noise, and use all the inputs, and one 8 wide hidden layer edit: works better with a sigmoid activation curve, but it converges more slowly
Yeh you're on the right track. Nice pattern emerges on this after 160 iterations.
http://playground.tensorflow.org/#activation=tanh&batchSize=...
Re: Tinker with a Neural Network in Your Browser
#40I started reading about ANNs in the 1980s, and had similar confusion to those here, since it was just for fun. I suggest reading a basic book or online information that goes over the basics [1]. I struggled through $200 text books, and jumped from one to the other as an autodidact. I am now studying TWEANNs (Topology and Weight Evolving Artificial Neural Networks), which basically are what you see here with the excep…
As far as the recent deep learning boom is concerned, genetic programming is really out of favor. I don't really see it in any of the deep learning (or even machine learning, for that matter) literature/successes/research groups.
"Neural networks" are a really really overloaded term. A ton of stuff referred to as "neural networks" has little to do with the "neural networks" that are used in the machine learning community.