Live data from Hacker News

Tinker with a Neural Network in Your Browser

playground.tensorflow.org

81–90 of 122 posts

Re: Tinker with a Neural Network in Your Browser

#81

When it says "right here in your browser," it's not joking. On my desktop (Safari), the window becomes unresponsive after a few iterations. Does not happen in Chrome. On my phone (Safari/iOS 9.3), the default neural nework doesn't converge at all even after 300 iterations while it does on the desktop, which is legit weird: https://i.imgur.com/KNaXeHH.png

I'm sorry you're having problems with Safari. I can't reproduce on my end, but if you're still having problems you can raise an issue on github with some information about your system.

Works perfectly for me on Safari 9.1 with no extensions

Re: Tinker with a Neural Network in Your Browser

#82

Has anyone been able to learn a function for the spiral (Swiss roll) data that's as good as a human-designed function would be?

For me key was to use x,y and either sin x, sin y or x squared y squared as inputs and 5 or 6 neurons in hidden layer.

Re: Tinker with a Neural Network in Your Browser

#83

When it says "right here in your browser," it's not joking. On my desktop (Safari), the window becomes unresponsive after a few iterations. Does not happen in Chrome. On my phone (Safari/iOS 9.3), the default neural nework doesn't converge at all even after 300 iterations while it does on the desktop, which is legit weird: https://i.imgur.com/KNaXeHH.png

Yeah, it's working great in Chrome on my Galaxy Tab 3!

Re: Tinker with a Neural Network in Your Browser

#85
post #75
post #69

Earlier quoted context omitted.

I'm not qualified to answer this, but I will anyway. To "operate" neural networks (as opposed to writing a framework for them), you need to know the building blocks. There are basic blocks like fully connected layers, convolutions, and nonlinear activations. Beyond those, there are higher level building blocks like LSTMs[1], gated recurrent units[2], highway layers[3], batch normalization[4], and residual blocks[5] t…

So you don't think some of these details will not be automated away in the near future so that it doesn't require a specialist to do operate a neural network?

Already, it's not nearly as hard as this demo makes it look. There's one recent advance in particular that isn't in this demo, and that is Batch Normalization.

If you've played around with it a bit, I'm sure you have seen that deeper layers are hard to train... You see the dashed lines representing signal in the network become weaker and weaker as the network gets deeper. BatchNorm works wonders with this. It takes statistics from the minibatch of training examples, and tries to normalize it so that the next layer gets input more similar to what it expects, even if the previous layer has changed. In practice you get a much better signal, so the network can learn a lot more efficiently.

Without BatchNorm, more than two hidden layers is tedious and error-prone to train. With it, you can train 10-12 layers easily. (With another recent advance, residual nets, you can train hundreds!)

Such advances pushes the limit for what you can train easily, and what still requires GSD ("graduate student descent", figuring out just the right parameters to get something to work through intuition, trial and error). You still have to watch out for overfitting, but the nice thing about that is that more training data helps.

Re: Tinker with a Neural Network in Your Browser

#86

When it says "right here in your browser," it's not joking. On my desktop (Safari), the window becomes unresponsive after a few iterations. Does not happen in Chrome. On my phone (Safari/iOS 9.3), the default neural nework doesn't converge at all even after 300 iterations while it does on the desktop, which is legit weird: https://i.imgur.com/KNaXeHH.png

I'm sorry you're having problems with Safari. I can't reproduce on my end, but if you're still having problems you can raise an issue on github with some information about your system.

[deleted]

Re: Tinker with a Neural Network in Your Browser

#87
post #28

I started reading about ANNs in the 1980s, and had similar confusion to those here, since it was just for fun. I suggest reading a basic book or online information that goes over the basics [1]. I struggled through $200 text books, and jumped from one to the other as an autodidact. I am now studying TWEANNs (Topology and Weight Evolving Artificial Neural Networks), which basically are what you see here with the excep…

Any thoughts on why genetic programming is not 'in fashion'? Does it have anything to do with complexity of the calculations?

I can imagine that the advanced models use many, many machines and only deliver results after a large training time. Genetic programming is not feasible then, if you cannot get a quick grasp of the potential results of a model.

Re: Tinker with a Neural Network in Your Browser

#88

Has anyone been able to learn a function for the spiral (Swiss roll) data that's as good as a human-designed function would be?

For this simple example just choosing the largest possible fully-connected network with ReLU and L2 regularization to prevent overfit quickly converges to a nice spiral (test loss of 0.001 for me):

http://playground.tensorflow.org/#activation=relu&regulariza...

Re: Tinker with a Neural Network in Your Browser

#89
post #83

When it says "right here in your browser," it's not joking. On my desktop (Safari), the window becomes unresponsive after a few iterations. Does not happen in Chrome. On my phone (Safari/iOS 9.3), the default neural nework doesn't converge at all even after 300 iterations while it does on the desktop, which is legit weird: https://i.imgur.com/KNaXeHH.png

Yeah, it's working great in Chrome on my Galaxy Tab 3!

To be honest, if it works in Chrome then it covers > 90% of people who would possibly be interested.

Re: Tinker with a Neural Network in Your Browser

#90

Has anyone been able to learn a function for the spiral (Swiss roll) data that's as good as a human-designed function would be?

For this simple example just choosing the largest possible fully-connected network with ReLU and L2 regularization to prevent overfit quickly converges to a nice spiral (test loss of 0.001 for me): http://playground.tensorflow.org/#activation=relu&regulariza...

Neat. How would the number of neurons N scale with the size of the spiral? (Size=number of turns)

Will N level off, meaning that it will really understand the structure of the spiral?

Post reply on HN