61–70 of 122 posts
This is great, but I think they should make it clear that this isn't using TensorFlow. From the title and domain I though they either had ported TF to Javascript(!) or we connecting to a server.
And great for challenging your friends in an epic battle of convergence!
This is a very cool toy. As someone with no experience in ML, this is an interesting visual approach to the absolute basics. And great for challenging your friends in an epic battle of convergence!
"Don’t Worry, You Can’t Break It. We Promise." (Nice, but it's completely unclear what's going on.)
Earlier quoted context omitted.
http://imgur.com/ypBQEWx Add some noise, and use all the inputs, and one 8 wide hidden layer edit: works better with a sigmoid activation curve, but it converges more slowly
Using all inputs and 6 layers of varying sizes. After about 500 iterations. http://i.imgur.com/x1MOpvl.jpg
This is great, but I think they should make it clear that this isn't using TensorFlow. From the title and domain I though they either had ported TF to Javascript(!) or we connecting to a server.
Wait - what it is using, then? I had assumed it was TF under Emscripten or similar.
As far as I can see the API[3] isn't much like TensorFlow.
[1] https://github.com/tensorflow/playground
[2] http://cs.stanford.edu/people/karpathy/convnetjs/
[3] https://github.com/tensorflow/playground/blob/master/nn.ts
Earlier quoted context omitted.
http://imgur.com/ypBQEWx Add some noise, and use all the inputs, and one 8 wide hidden layer edit: works better with a sigmoid activation curve, but it converges more slowly
> Add some noise This actually makes the dataset harder to fit to. It is not the same thing here as the "training with noise" method where random noise would be added to each batch, as an alternative means of Tikhonov regularization.
The noise doesn't go far enough to start confusing points between different clusters, but it adds more points.
That said, my knowledge of neural nets is fairly limited.
I'm not well versed in neural networks but a lot of the new neural network software stacks coming out seem to be quite plug and plug. What kind of expertise would engineers need to have a few years from now when the technology is well developed and it doesn't need to be rewritten from scratch every time?
To "operate" neural networks (as opposed to writing a framework for them), you need to know the building blocks. There are basic blocks like fully connected layers, convolutions, and nonlinear activations. Beyond those, there are higher level building blocks like LSTMs[1], gated recurrent units[2], highway layers[3], batch normalization[4], and residual blocks[5] that are made up of simpler blocks. Learning what these do and when it's appropriate to use them requires following current literature.
Operating neural networks requires some systems engineering skill. It takes a long time to train a single network and you'll find yourself trying many different architectures and hyperparameters along the way. Because of this, you'll want to distribute the training across many different systems and be able to easily monitor and deploy jobs on those systems.
A solid grasp of mathematics is useful to effectively debug your networks. You'll frequently find your network doesn't converge or gives totally garbage results, so you need to know how to dig into the network internals and understand how everything works. This is especially true if you're implementing a new building block from a paper.
Finally, know your machine learning and statistics fundamentals. Understand overfitting, model capacity, cross validation, probability, model ensembles, information theory, and so on. Know when a simpler model is more appropriate.
[1] ftp://ftp.idsia.ch/pub/juergen/fki-207-95.ps.gz
[2] http://arxiv.org/abs/1409.1259
[3] http://arxiv.org/abs/1505.00387
I started reading about ANNs in the 1980s, and had similar confusion to those here, since it was just for fun. I suggest reading a basic book or online information that goes over the basics [1]. I struggled through $200 text books, and jumped from one to the other as an autodidact. I am now studying TWEANNs (Topology and Weight Evolving Artificial Neural Networks), which basically are what you see here with the excep…
I brainstormed for a while about using genetic algorithms to decide the network topology. I'm glad someone else invented that already! Less work for me to do now.