When it says "right here in your browser," it's not joking. On my desktop (Safari), the window becomes unresponsive after a few iterations. Does not happen in Chrome. On my phone (Safari/iOS 9.3), the default neural nework doesn't converge at all even after 300 iterations while it does on the desktop, which is legit weird: https://i.imgur.com/KNaXeHH.png
I'm sorry you're having problems with Safari. I can't reproduce on my end, but if you're still having problems you can raise an issue on github with some information about your system.
Tinker with a Neural Network in Your Browser
81–90 of 122 posts
Re: Tinker with a Neural Network in Your Browser
#82Has anyone been able to learn a function for the spiral (Swiss roll) data that's as good as a human-designed function would be?
Re: Tinker with a Neural Network in Your Browser
#83When it says "right here in your browser," it's not joking. On my desktop (Safari), the window becomes unresponsive after a few iterations. Does not happen in Chrome. On my phone (Safari/iOS 9.3), the default neural nework doesn't converge at all even after 300 iterations while it does on the desktop, which is legit weird: https://i.imgur.com/KNaXeHH.png
Re: Tinker with a Neural Network in Your Browser
#84Has anyone been able to learn a function for the spiral (Swiss roll) data that's as good as a human-designed function would be?
Re: Tinker with a Neural Network in Your Browser
#85Earlier quoted context omitted.
I'm not qualified to answer this, but I will anyway. To "operate" neural networks (as opposed to writing a framework for them), you need to know the building blocks. There are basic blocks like fully connected layers, convolutions, and nonlinear activations. Beyond those, there are higher level building blocks like LSTMs[1], gated recurrent units[2], highway layers[3], batch normalization[4], and residual blocks[5] t…
So you don't think some of these details will not be automated away in the near future so that it doesn't require a specialist to do operate a neural network?
If you've played around with it a bit, I'm sure you have seen that deeper layers are hard to train... You see the dashed lines representing signal in the network become weaker and weaker as the network gets deeper. BatchNorm works wonders with this. It takes statistics from the minibatch of training examples, and tries to normalize it so that the next layer gets input more similar to what it expects, even if the previous layer has changed. In practice you get a much better signal, so the network can learn a lot more efficiently.
Without BatchNorm, more than two hidden layers is tedious and error-prone to train. With it, you can train 10-12 layers easily. (With another recent advance, residual nets, you can train hundreds!)
Such advances pushes the limit for what you can train easily, and what still requires GSD ("graduate student descent", figuring out just the right parameters to get something to work through intuition, trial and error). You still have to watch out for overfitting, but the nice thing about that is that more training data helps.
Re: Tinker with a Neural Network in Your Browser
#86When it says "right here in your browser," it's not joking. On my desktop (Safari), the window becomes unresponsive after a few iterations. Does not happen in Chrome. On my phone (Safari/iOS 9.3), the default neural nework doesn't converge at all even after 300 iterations while it does on the desktop, which is legit weird: https://i.imgur.com/KNaXeHH.png
I'm sorry you're having problems with Safari. I can't reproduce on my end, but if you're still having problems you can raise an issue on github with some information about your system.
Re: Tinker with a Neural Network in Your Browser
#87I started reading about ANNs in the 1980s, and had similar confusion to those here, since it was just for fun. I suggest reading a basic book or online information that goes over the basics [1]. I struggled through $200 text books, and jumped from one to the other as an autodidact. I am now studying TWEANNs (Topology and Weight Evolving Artificial Neural Networks), which basically are what you see here with the excep…
I can imagine that the advanced models use many, many machines and only deliver results after a large training time. Genetic programming is not feasible then, if you cannot get a quick grasp of the potential results of a model.
Re: Tinker with a Neural Network in Your Browser
#88Has anyone been able to learn a function for the spiral (Swiss roll) data that's as good as a human-designed function would be?
http://playground.tensorflow.org/#activation=relu®ulariza...
Re: Tinker with a Neural Network in Your Browser
#89When it says "right here in your browser," it's not joking. On my desktop (Safari), the window becomes unresponsive after a few iterations. Does not happen in Chrome. On my phone (Safari/iOS 9.3), the default neural nework doesn't converge at all even after 300 iterations while it does on the desktop, which is legit weird: https://i.imgur.com/KNaXeHH.png
Yeah, it's working great in Chrome on my Galaxy Tab 3!
Re: Tinker with a Neural Network in Your Browser
#90Has anyone been able to learn a function for the spiral (Swiss roll) data that's as good as a human-designed function would be?
For this simple example just choosing the largest possible fully-connected network with ReLU and L2 regularization to prevent overfit quickly converges to a nice spiral (test loss of 0.001 for me): http://playground.tensorflow.org/#activation=relu®ulariza...
Will N level off, meaning that it will really understand the structure of the spiral?