Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

dl.acm.org

Post reply on HN
Print view View original

21–30 of 58 posts

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#21

Post by [deleted] »

raverbashing wrote: post #15
vengarioth wrote: post #13

Earlier quoted context omitted.

I don't think you can generalize it this way, ASTs could be compiled to fast machine code, also it really depends on the solutions the algorithms come up with. The NN is bound to its number of parameters, while the Genetic's program varies in length and can become quite small if length is part of the fitness function.

Well, the accrual of "useless code" (there's a name for this that I forgot") is a known problem, but it is also something that stabilizes the learning process I don't think it's as simple as putting the length of the AST in the goal function (but it's something interesting to try). Depending on compile speed vs running speed you might be better off interpreting your ASTs

[deleted]

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#22

Post by [deleted] »

raverbashing wrote: post #15
vengarioth wrote: post #13

Earlier quoted context omitted.

I don't think you can generalize it this way, ASTs could be compiled to fast machine code, also it really depends on the solutions the algorithms come up with. The NN is bound to its number of parameters, while the Genetic's program varies in length and can become quite small if length is part of the fitness function.

Well, the accrual of "useless code" (there's a name for this that I forgot") is a known problem, but it is also something that stabilizes the learning process I don't think it's as simple as putting the length of the AST in the goal function (but it's something interesting to try). Depending on compile speed vs running speed you might be better off interpreting your ASTs

[deleted]

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#23

Post by posterboy »

phoe-krk wrote: post #10
aqsalose wrote: post #7

Earlier quoted context omitted.

They are comparing their genetic programming results with a deep learning paper published in 2015. [1] [1] https://www.nature.com/nature/journal/v518/n7540/abs/nature1...

What a time to live in, when papers from 2015 are "really old" in 2017.

What is "exponential growth"!

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#24

Post by symmetricsaurus »

bomdo wrote: post #17

I was a little surprised at the headline, since I expected 'outperforms' to mean that it had better end-results, which is of course not the case. GP is just much faster due to it's relative simplicity and the results are close enough to those achieved with NN and deep learning. > Finally, while generally matching the skill level of controllers from neuro-evolution/deep learning, the genetic programming solutions evol…

> I was a little surprised at the headline, since I expected 'outperforms' to mean that it had better end-results, which is of course not the case. GP is just much faster due to it's relative simplicity and the results are close enough to those achieved with NN and deep learning.

From figure 3 in the paper it seems like it outperforms DQN on all games but one. So, it has better end results as well.

Edit: There are other results linked in this thread that are better than the 2015 DQN results that the paper refers to.

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#25

Post by [deleted] »

kensoh wrote: post #18
smdz wrote: post #9

One of the huge benefits of GPs over NNs is the ease of reverse engineering a GP tree compared to NN models. Its not effortless however. Its just not mathematically complex like NNs i.e. a programmer who isn't a mathematician can analyze GPs with a lot of patience EDIT: I have found GPs to be relatively slow-to-very-slow. But very likely that is because of the lack of interest and development compared to NNs

Would some form of computer-aided tool be able to help the programmer analyze GPs? It sounds like there's some repetitive process when you say 'GPs with a lot of patience'. I was thinking along the line that if such a tool is possible, then it might be possible to let the development and evolution of GPs iterate faster than deep learning. I found this thread really interesting, because of the small size, such impleme…

[deleted]

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#27

Post by [deleted] »

symmetricsaurus wrote: post #24
bomdo wrote: post #17

I was a little surprised at the headline, since I expected 'outperforms' to mean that it had better end-results, which is of course not the case. GP is just much faster due to it's relative simplicity and the results are close enough to those achieved with NN and deep learning. > Finally, while generally matching the skill level of controllers from neuro-evolution/deep learning, the genetic programming solutions evol…

> I was a little surprised at the headline, since I expected 'outperforms' to mean that it had better end-results, which is of course not the case. GP is just much faster due to it's relative simplicity and the results are close enough to those achieved with NN and deep learning. From figure 3 in the paper it seems like it outperforms DQN on all games but one. So, it has better end results as well. Edit: There are ot…

[deleted]

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#28

Post by [deleted] »

symmetricsaurus wrote: post #24
bomdo wrote: post #17

I was a little surprised at the headline, since I expected 'outperforms' to mean that it had better end-results, which is of course not the case. GP is just much faster due to it's relative simplicity and the results are close enough to those achieved with NN and deep learning. > Finally, while generally matching the skill level of controllers from neuro-evolution/deep learning, the genetic programming solutions evol…

> I was a little surprised at the headline, since I expected 'outperforms' to mean that it had better end-results, which is of course not the case. GP is just much faster due to it's relative simplicity and the results are close enough to those achieved with NN and deep learning. From figure 3 in the paper it seems like it outperforms DQN on all games but one. So, it has better end results as well. Edit: There are ot…

[deleted]

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#29

Post by cshenton »

This is super cool, but it doesn't outperform deep learning based RL methods.

In fact, I'm not sure how much more compute efficient than something like A3C it would be. That can produce 4x the score of DQN in a comparable number of hours (and on a CPU).

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#30

Post by pmoriarty »

smdz wrote: post #9

One of the huge benefits of GPs over NNs is the ease of reverse engineering a GP tree compared to NN models. Its not effortless however. Its just not mathematically complex like NNs i.e. a programmer who isn't a mathematician can analyze GPs with a lot of patience EDIT: I have found GPs to be relatively slow-to-very-slow. But very likely that is because of the lack of interest and development compared to NNs

"I have found GPs to be relatively slow-to-very-slow."

There are two different speeds one could measure in regards to GP's.

The first is the speed at which one evolves solutions. This can, in fact, be frightfully slow and eat up all the hardware you can throw at it (depending on how large your populations are, how fast your fitness function is, etc).

The second is the speed of the evolved solution. This may be slow, but doesn't have to be. In fact, the speed of the solution could be part of what's being evolved. So you could explicitly evolve something fast, if you wanted (or it might just wind up being fast by chance).

One could also take an evolved solution, analyze it, and then optimize it or rewrite it using the insights you got from your analysis. That could be even faster.

Post reply on HN