I've written a NEAT implementation in Rust a few years back. It was able to train XOR and pole balancing.
https://sgolem.com/blog/neat-simple-neuroevolution-framework...
Neuroevolution of augmenting topologies (NEAT algorithm)
11–20 of 34 posts
Re: Neuroevolution of augmenting topologies (NEAT algorithm)
#12For anyone interested in NEAT, you’ll likely enjoy Ken’s later work on novelty search, open-endedness, etc.
His book “Why Greatness Cannot be Planned: The Myth of the Objectives” is one of the most perspective altering works I have ever read.
Very excited to see what he does next. He mentioned on Twitter a couple times his interest in representation learning and how objective based search affects this. Very interesting stuff
Re: Neuroevolution of augmenting topologies (NEAT algorithm)
#13I post a link to NEAT here about once a week. The big problem is that NEAT can't leverage a GPU effectively at scale (arbitrary topologies vs bipartite graphs) Other than that, it feels like a "road not taken" of machine learning. It handles building complex agent behaviors really well, and pressure to minimal topology results in understandable and reverse interpretable networks. Its easier to learn and implement tha…
> The big problem is that NEAT can't leverage a GPU effectively at scale (arbitrary topologies vs bipartite graphs) Is that true? These graphs can be transformed into a regular tensor shape with zero weights on unused connections. If you were worried about too much time/space used by these zero weights, you could introduce parsimony pressure related to the dimensions of transformed tensors rather than on the bipartit…
Or even use CuSparse, if you don't mind a little bit of extra work over normal cudnn.
Re: Neuroevolution of augmenting topologies (NEAT algorithm)
#14Legendary Sethbling video from 2015 where he implemented NEAT for SMW: https://www.youtube.com/watch?v=qv6UVOQ0F44&t=1
Re: Neuroevolution of augmenting topologies (NEAT algorithm)
#15NEAT:
https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_t...
NEAT > See also links to EANT.
EANT: https://en.wikipedia.org/wiki/Evolutionary_acquisition_of_ne... :
> Evolutionary acquisition of neural topologies (EANT/EANT2) is an evolutionary reinforcement learning method that evolves both the topology and weights of artificial neural networks. It is closely related to the works of Angeline et al. [1] and Stanley and Miikkulainen. [2 [NEAT (2002)] Like the work of Angeline et al., the method uses a type of parametric mutation that comes from evolution strategies and evolutionary programming (now using the most advanced form of the evolution strategies CMA-ES in EANT2), in which adaptive step sizes are used for optimizing the weights of the neural networks. Similar to the work of Stanley (NEAT), the method starts with minimal structures which gain complexity along the evolution path.
(in Hilbert space)
> [EANT] introduces a genetic encoding called common genetic encoding (CGE) that handles both direct and indirect encoding of neural networks within the same theoretical framework. The encoding has important properties that makes it suitable for evolving neural networks:
> It is complete in that it is able to represent all types of valid phenotype networks.
> It is closed, i.e. every valid genotype represents a valid phenotype. (Similarly, the encoding is closed under genetic operators such as structural mutation and crossover.)
> These properties have been formally proven. [3]
Neither MOSES: Meta-Optimizing Semantic Evolutionary Search nor PLN: Probabilistic Logic Networks are formally proven FWIU.
/? Z3 ... https://news.ycombinator.com/item?id=41944043 :
> Does [formal verification [with Z3]] distinguish between xy+z and z+yx, in terms of floating point output?
> math.fma() would be a different function call with the same or similar output.
> (deal-solver is a tool for verifying formal implementations in Python with Z3.)
Re: Neuroevolution of augmenting topologies (NEAT algorithm)
#16Earlier quoted context omitted.
> The big problem is that NEAT can't leverage a GPU effectively at scale I worry how much experimentation we are leaving on the table simply because these paths wouldn't implicate nvidia stockholders in an advantageous manner. Modern CPUs might as well be GPUs when you are running GA experiments from the 80s and 90s on them. We can do millions of generations per hour and populations that span many racks of machines w…
The more immediate causal reason is that there is no off the shelf high performance hardware to accelerate these paths. GPUs were made for graphics, not machine learning. They just happened to be really good at running certain kinds of models that can be reduced to a ton of matrix math that GPUs do really really fast. Something like Intel Xeon Phi or one of the many-many-core ARM designs I've heard talked about for y…
> When I was in college in the early 2000s I played around with genetic algorithms and artificial life simulations on a cluster of IBM Cell Broadband Engine processors
Early/Mid/Late 2000's I was working on a hobby project for artificial life with ga evolved brains. I spent a lot of time investigating options, like the cell processor (I bought a playstation to test with). I also looked at interesting multi-core cpu's that were being introduced.
I ended up with a combo of CPU+GPU on networked PC's, which was better than nothing but not ideal.
Re: Neuroevolution of augmenting topologies (NEAT algorithm)
#17I post a link to NEAT here about once a week. The big problem is that NEAT can't leverage a GPU effectively at scale (arbitrary topologies vs bipartite graphs) Other than that, it feels like a "road not taken" of machine learning. It handles building complex agent behaviors really well, and pressure to minimal topology results in understandable and reverse interpretable networks. Its easier to learn and implement tha…
> The big problem is that NEAT can't leverage a GPU effectively at scale (arbitrary topologies vs bipartite graphs) Is that true? These graphs can be transformed into a regular tensor shape with zero weights on unused connections. If you were worried about too much time/space used by these zero weights, you could introduce parsimony pressure related to the dimensions of transformed tensors rather than on the bipartit…
We can also show that sparse NNs under some conditions are ensembles of discrete subnets, and the authors of the original dropout paper argue that [dropout] effectively creates something akin to a forest of subnets all in "superposition".
Re: Neuroevolution of augmenting topologies (NEAT algorithm)
#18Earlier quoted context omitted.
The more immediate causal reason is that there is no off the shelf high performance hardware to accelerate these paths. GPUs were made for graphics, not machine learning. They just happened to be really good at running certain kinds of models that can be reduced to a ton of matrix math that GPUs do really really fast. Something like Intel Xeon Phi or one of the many-many-core ARM designs I've heard talked about for y…
> When I was in college in the early 2000s I played around with genetic algorithms and artificial life simulations on a cluster of IBM Cell Broadband Engine processors Early/Mid/Late 2000's I was working on a hobby project for artificial life with ga evolved brains. I spent a lot of time investigating options, like the cell processor (I bought a playstation to test with). I also looked at interesting multi-core cpu's…
> I ended up with a combo of CPU+GPU on networked PC's, which was better than nothing but not ideal.
The biggest constraint I've seen with scaling up these simulations is maintaining coherence of population dynamics across all of the processing units. The less often population members are exchanged, the more likely you will wind up with a decoupled population and stuck in a ditch somewhere. Since modern CPUs can go so damn fast, you need to exchange members quite frequently. Memory bandwidth and latency are the real troublemakers.
You can spread a simulation across many networked nodes, but then your effective cycle time is potentially millions of times greater than if you keep it all in one socket. I think the newest multi-die CPUs hit the perfect sweet spot for these techniques (~100ns latency domain).
Re: Neuroevolution of augmenting topologies (NEAT algorithm)
#19HyperNEAT is also really cool: https://en.wikipedia.org/wiki/HyperNEAT
It uses NEAT to evolve a CPPN which is then sampled at some specified resolution to determine the actual network topology. The really cool thing is the sampling resolution can be varied to scale the size of the network while maintaining the overall "shape" of the topology. I took Ken's neuroevolution course back in the day and worked with HyperNEAT before deep learning got big. Ever since, deep learning network architectures have always reminded me of the dense layered topologies that result from higher sampling resolutions with HyperNEAT. It would be interesting to see if HyperNEAT along with Ken's novelty search technique could be used to evolve useful deep learning architectures.
Re: Neuroevolution of augmenting topologies (NEAT algorithm)
#20I post a link to NEAT here about once a week. The big problem is that NEAT can't leverage a GPU effectively at scale (arbitrary topologies vs bipartite graphs) Other than that, it feels like a "road not taken" of machine learning. It handles building complex agent behaviors really well, and pressure to minimal topology results in understandable and reverse interpretable networks. Its easier to learn and implement tha…
> The big problem is that NEAT can't leverage a GPU effectively at scale (arbitrary topologies vs bipartite graphs) Is that true? These graphs can be transformed into a regular tensor shape with zero weights on unused connections. If you were worried about too much time/space used by these zero weights, you could introduce parsimony pressure related to the dimensions of transformed tensors rather than on the bipartit…
Another plausible strategy to neutralize arbitrary topologies: compile individual solutions or groups of similar solutions into big compute shaders that execute the network and evaluate expensive fitness functions, with parallel execution over multiple test cases (aggregated in postprocessing) and/or over different numerical parameters for the same topology.