21–30 of 122 posts
Neat stuff, fun to play with. I wasn't able to get a net to classify the swiss roll. Last time I was playing around with this stuff I found the single biggest factor in the success was the optimizer used. Is this just using a simple gradient descent? I would like to see a drop down for different optimizers.
Add some noise, and use all the inputs, and one 8 wide hidden layer
edit: works better with a sigmoid activation curve, but it converges more slowly
Can somebody explain what I'm watching when I press play?
Or perhaps explain how all the different inputs influence the result? I more or less get that it's just iterating over the data to approximate the given data set when you press play but I have no idea how giving it more or less neurons changes that, to name an example.
The neural network is essentially the nodes in the middle, linked together by various weights. During training, the test data points are fed forward into the network, creating an output. That output is then fed backward using something called "back propagation" which is used to adjust the weights.
Typically, the more hidden layers or nodes per layer, the more difficult gradients that can be learned. Zero hidden layers essentially forms a linear gradient that can only be used to split very basic, linearly-separable data (drawing a straight line to separate the different types)
Neural networks have lots of little knobs and levers you can adjust. That's what all these inputs are that you see.
(Nice, but it's completely unclear what's going on.)
Can somebody explain what I'm watching when I press play?
Think of the whole neural net as a function:
input * weight = output
At each iteration, we feed in the input to the neural net. Then the neural net compares what output it gets to the correct output.
For example, input1 is 5, and the correct output for input1 should have been 2. But the neural net got 3 as the output. So it then decreases the weights slightly so it would get 2.75 next time it has input of 5. Repeat thousands of times. That's the basic idea for machine learning and neural networks.
The algorithm it uses to figure out how much to decrease the weights is called "backpropagation" which uses gradient descent. To explain gradient descent, as as a roller coaster track. Imagine the roller coaster starts off on a random location on the track. Then gravity takes the roller coaster down the track until it ends up on a low point between two hills and stays there. This is the new location of the roller coaster. This new location is nice because it has the lowest energy the roller coaster could find, so it stays there. (We use derivatives to figure out the slope of a curve, which then gives us the direction where the curve goes downhill).
In neural networks, the roller coaster curve is the "cost function", which basically calculates the amount of difference between the neural net's output and the actual correct output it should have got. The initial weight is the roller coaster's initial position. The new weight is the roller coaster's final position, at the bottom of the cost function curve. This new position thus gives us the lowest cost.
Note that there may be even lower valleys, but when we roll the rollercoaster it stops at its nearest low valley. This is why we randomize the weights at the beginning - to put the roller coaster near possibly even lower valleys.
Can somebody explain what I'm watching when I press play?
Yeah I feel like we need some decent understanding of neural networks to have more context on this. Its kind of like being given a specialized shovel but not knowing why you need it or why you should dig holes.
If you're expecting a lesson, you'll likely be disappointed, but I think there's real value in a true playground.
I think the biggest improvement would be if, when hovering over a 'neuron', you get a visual representation of what feeds into it.
I wish they had more interesting data sets.
[1] http://natureofcode.com/book/chapter-10-neural-networks/
[2] http://www.genetic-programming.comNeat stuff, fun to play with. I wasn't able to get a net to classify the swiss roll. Last time I was playing around with this stuff I found the single biggest factor in the success was the optimizer used. Is this just using a simple gradient descent? I would like to see a drop down for different optimizers.
I don't know if that's a general feature to need fewer neurons with each layer, but that seems to work here.
Can somebody explain what I'm watching when I press play?
Hopefully this helps (correct me if I'm wrong, I'm still learning about neural nets): Think of the whole neural net as a function: input * weight = output At each iteration, we feed in the input to the neural net. Then the neural net compares what output it gets to the correct output. For example, input1 is 5, and the correct output for input1 should have been 2. But the neural net got 3 as the output. So it then dec…