Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

dl.acm.org

Post reply on HN
Print view View original

51–58 of 58 posts

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#51

Post by sdenton4 »

raverbashing wrote: post #15
vengarioth wrote: post #13

Earlier quoted context omitted.

I don't think you can generalize it this way, ASTs could be compiled to fast machine code, also it really depends on the solutions the algorithms come up with. The NN is bound to its number of parameters, while the Genetic's program varies in length and can become quite small if length is part of the fitness function.

Well, the accrual of "useless code" (there's a name for this that I forgot") is a known problem, but it is also something that stabilizes the learning process I don't think it's as simple as putting the length of the AST in the goal function (but it's something interesting to try). Depending on compile speed vs running speed you might be better off interpreting your ASTs

That's kinda horrifying... Accruing useless code to reduce the probability of mutating the useful code? Sounds like a better regularization strategy is needed.

I'm generally pretty suspicious of generic algorithms; why take a random walk when you can March along the gradient towards a solution?

It might be interesting to try using GA for neural architecture, though, and gradient descent to train the network... (Though it sounds expensive.)

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#52

Post by hchasestevens »

gourou wrote: post #41

Genetic Programming seems lightweight, what are some cool applications they have?

There are actually lots of very exciting GP applications! One of my favorites is "Fixing 55 out of 105 bugs for $8 each", in which GP is used to automatically repair code: https://www.cs.virginia.edu/~weimer/p/weimer-icse2012-genpro... . They've also achieved better-than-human level results in antenna design for NASA ( https://ti.arc.nasa.gov/m/pub-archive/1244h/1244%20(Hornby).... ) and in discovering novel quantum computing algorithms ( http://faculty.hampshire.edu/lspector/pubs/GP-quantum-GP98-w... ).

I'll also shamelessly hock here my GP framework for Python, in case you're interested in experimenting: https://github.com/hchasestevens/monkeys

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#53

Post by raverbashing »

sdenton4 wrote: post #51
raverbashing wrote: post #15

Earlier quoted context omitted.

Well, the accrual of "useless code" (there's a name for this that I forgot") is a known problem, but it is also something that stabilizes the learning process I don't think it's as simple as putting the length of the AST in the goal function (but it's something interesting to try). Depending on compile speed vs running speed you might be better off interpreting your ASTs

That's kinda horrifying... Accruing useless code to reduce the probability of mutating the useful code? Sounds like a better regularization strategy is needed. I'm generally pretty suspicious of generic algorithms; why take a random walk when you can March along the gradient towards a solution? It might be interesting to try using GA for neural architecture, though, and gradient descent to train the network... (Thoug…

> why take a random walk when you can March along the gradient towards a solution?

Because your problem has no smooth/continuous gradient

Because your problem has a giant search space

Because your problem can do with a "close enough" solution

Try gradient descending a symbolic regression and we'll talk

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#54

Post by habitue »

cshenton wrote: post #44
habitue wrote: post #42

Earlier quoted context omitted.

A3C is only ever run on one game at a time[0]. This paper gets good performance on all games with the same agent [0] read as: I have only seen papers with 1 agent per game for A3C

So it can train on one game and play without training on a previously unseen (but also atari) game? That's pretty neat, DQN and A3C certainly can't do that.

No, in this case it is trained on all the games, but retains good scores on all of them. If you train basic AC3 in all the games, you'll get poor performance on all the games due to catastrophic forgetting

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#57

Post by uoaei »

tdb7893 wrote: post #55
gourou wrote: post #43

What's a good starting point for someone interested in building game AI?

Current game ai is vastly different than this I think. I think there is a good writeup on the ai from FEAR that might be a decent read

http://alumni.media.mit.edu/~jorkin/goap.html

Re: Multi-Task Learning in Atari Video Games with Emergent Tangled Program Graphs

#58

Post by posterboy »

nocoder wrote: post #2

This sounds interesting. I will like someone from the field of genetic programming on how this works and how it differs from current DL approaches.

In my rough understanding, DNN is genetic programming i a sense, matrixes over a vectorfield can be thought of as operators, and neurons are layers of xor circuits ...
Post reply on HN