Tinker with a Neural Network in Your Browser

Post by eggy » Thu, Apr 14, 2016, 12:16 PM UTC

daveguy wrote: post #101

DavidSJ wrote: post #98

Earlier quoted context omitted.

If your program is a neural network with N parameters, or a program tree with N nodes, then testing against data takes O(N) time. With evolutionary computation, what you get for your trouble is a single real number -- the loss: how bad it did. With neural networks, backpropagation gives you N real numbers: the gradient of loss with respect to each parameter. Put another way: with evolution you have to stumble around…

> It might take 4.54 billion years. And then only if you have a system powerful enough to accurately simulate a planet full of molecules. Although I do think there is a balance between GA and structured NN which will lead to faster and better results than the deep NN alone. We already see some of the best deep NNs incorporating specific structures.

I think neural networks and other forms of evolutionary computation will merge as I have been writing in my other replies in this thread. TWEANNs incorporate EC into evolving ANNs. The other article I cited above on soil mechanics, beat out expert systems, ANNs, statistics, and used GP. MEP, or Multi-Expression Programming for GP incorporates being able to put more than one solution into a gene without increasing the processing times thereby overcoming the inefficiencies of 1990s-era GP. Here is a recent article using it that is not behind a paywall or via sci-hub.io [1]. It needs better editing, but there are other references if you search for Multi-expression Genetic Programming.

  [1] http://benthamopen.com/ABSTRACT/TOPEJ-9-21

Post by conceit » Thu, Apr 14, 2016, 7:50 PM UTC

romaniv wrote: post #112

erostrate wrote: post #94

The swiss roll problem also illustrates nicely the idea behind deep learning. Before deep learning people would manually design all these extra features sin(x_1), x_1^2, etc. because they thought it was necessary to fit this swiss roll dataset. So they would use a shallow network with all these features like this: http://imgur.com/H1cvt8d Then the deep learning guys realized that you don't have to engineer all these…

>Before deep learning people would manually design all these extra features sin(x_1), x_1^2, etc. It's probably worth pointing out that this is true for ANNs, but there were (and are) other "shallow" classifiers that can handle swiss roll problem without manual parameter encoding. SVMs, for example. http://cs.stanford.edu/people/karpathy/svmjs/demo/

needs another image link for visualization

Re: Tinker with a Neural Network in Your Browser

Re: Tinker with a Neural Network in Your Browser