11–20 of 58 posts
Earlier quoted context omitted.
They are comparing their genetic programming results with a deep learning paper published in 2015. [1] [1] https://www.nature.com/nature/journal/v518/n7540/abs/nature1...
What a time to live in, when papers from 2015 are "really old" in 2017.
One of the huge benefits of GPs over NNs is the ease of reverse engineering a GP tree compared to NN models. Its not effortless however. Its just not mathematically complex like NNs i.e. a programmer who isn't a mathematician can analyze GPs with a lot of patience EDIT: I have found GPs to be relatively slow-to-very-slow. But very likely that is because of the lack of interest and development compared to NNs
I've dabbled in GP and I really like it but those ASTs can get huge if they're not carefully pruned and might not add to the solution at all.
One of the huge benefits of GPs over NNs is the ease of reverse engineering a GP tree compared to NN models. Its not effortless however. Its just not mathematically complex like NNs i.e. a programmer who isn't a mathematician can analyze GPs with a lot of patience EDIT: I have found GPs to be relatively slow-to-very-slow. But very likely that is because of the lack of interest and development compared to NNs
Partly, I think they're also fundamentally slower than NN because you're manipulating and executing programs (ASTs[1]) while for NNs you just adjust some values. I've dabbled in GP and I really like it but those ASTs can get huge if they're not carefully pruned and might not add to the solution at all. [1] https://en.wikipedia.org/wiki/Abstract_syntax_tree
Earlier quoted context omitted.
They are comparing their genetic programming results with a deep learning paper published in 2015. [1] [1] https://www.nature.com/nature/journal/v518/n7540/abs/nature1...
What a time to live in, when papers from 2015 are "really old" in 2017.
Earlier quoted context omitted.
Partly, I think they're also fundamentally slower than NN because you're manipulating and executing programs (ASTs[1]) while for NNs you just adjust some values. I've dabbled in GP and I really like it but those ASTs can get huge if they're not carefully pruned and might not add to the solution at all. [1] https://en.wikipedia.org/wiki/Abstract_syntax_tree
I don't think you can generalize it this way, ASTs could be compiled to fast machine code, also it really depends on the solutions the algorithms come up with. The NN is bound to its number of parameters, while the Genetic's program varies in length and can become quite small if length is part of the fitness function.
I don't think it's as simple as putting the length of the AST in the goal function (but it's something interesting to try).
Depending on compile speed vs running speed you might be better off interpreting your ASTs
This sounds interesting. I will like someone from the field of genetic programming on how this works and how it differs from current DL approaches.
Kelly's approach involves evolving teams of programs. His basic strategy is to have a scalable problem decomposition strategy. So programs that process pixels and the teaming of those programs are grouped together. The groupings (teams) themselves are co-evolved with the programs, simultaneously. This enables niching and specialization behavior. This builds on earlier work on 'symbiotic bid-based genetic programming'…
Ding ding ding. This is where the money is at, good yet cheap sensors that sense human level actions are needed for IoT to be impactful.
> Finally, while generally matching the skill level of controllers from neuro-evolution/deep learning, the genetic programming solutions evolved here are several orders of magnitude simpler, resulting in real-time operation at a fraction of the cost.
> Moreover, TPG solutions are particularly elegant, thus supporting real-time operation without specialized hardware
This is the key takeaway and yet another reminder to not make deep learning the hammer for all your fuzzy problems.
One of the huge benefits of GPs over NNs is the ease of reverse engineering a GP tree compared to NN models. Its not effortless however. Its just not mathematically complex like NNs i.e. a programmer who isn't a mathematician can analyze GPs with a lot of patience EDIT: I have found GPs to be relatively slow-to-very-slow. But very likely that is because of the lack of interest and development compared to NNs
This sounds interesting. I will like someone from the field of genetic programming on how this works and how it differs from current DL approaches.
Kelly's approach involves evolving teams of programs. His basic strategy is to have a scalable problem decomposition strategy. So programs that process pixels and the teaming of those programs are grouped together. The groupings (teams) themselves are co-evolved with the programs, simultaneously. This enables niching and specialization behavior. This builds on earlier work on 'symbiotic bid-based genetic programming'…
I am wondering, whether a similar approach is possible with current DL models and will it have any performance improvements over what is existing or whether it will be computationally even more expensive.
Earlier quoted context omitted.
I don't think you can generalize it this way, ASTs could be compiled to fast machine code, also it really depends on the solutions the algorithms come up with. The NN is bound to its number of parameters, while the Genetic's program varies in length and can become quite small if length is part of the fitness function.
Well, the accrual of "useless code" (there's a name for this that I forgot") is a known problem, but it is also something that stabilizes the learning process I don't think it's as simple as putting the length of the AST in the goal function (but it's something interesting to try). Depending on compile speed vs running speed you might be better off interpreting your ASTs