Live data from Hacker News

NeuroEvolution – Flappy Bird

xviniette.github.io

61–70 of 103 posts

Re: NeuroEvolution – Flappy Bird

#61
post #15

Earlier quoted context omitted.

It is a using a genetic algorithm so learning lies in the mutation and crossover of chromosomes as well as selection of the fittest individuals. https://github.com/xviniette/FlappyLearning/blob/gh-pages/Ne...

It's also using a simple neural network, which is the target of the genetic algorithm, if I understood it right. I haven't seen this combination often - does that make sense in general, or is this just interesting as in playing around with those concepts?

it's a pretty standard procedure to train NNs through GAs but usually not very efficient (e.g. compared to backtracking).

in some cases you might lack an easy way to calculate a fitness score out of the NN performance, which is needed to run the GA.

i tried training a simple NN with a stupid hill climber some time ago but quickly hit a roadblock even with very few neurons because of local minima ... or maybe bugs.

i guess for more complicated problems the pure GA training method might just not be "cost effective" (i.e. time/quality tradeoff).

Re: NeuroEvolution – Flappy Bird

#62
post #53
post #39

This is one of these fantastic "all that is done with so little code?!" moments. Amazing that this can be done in 300 lines of perfectly readable JS without any libraries. And the author apparently wrote it in 2 days. Great intro to genetic algorithms and reinforcement learning. Universities should teach like that. Edit: Somebody has asked about the big picture how the thing works: You use a neuronal network to decid…

These are the exactly gems that I am constantly in search of, and thrilled every time I come across one. This is going to turn out to be a rant, but sometimes it feels like there's a culture of solving problems by a philosophy of "let's throw more money at it, more technology at it, more people, more unnecessary abstraction layers, complexity and bloat - it's bound to be solved eventually". And with enough fire power…

Sounds like you are noting that brute force is a very effective technique. Combined with explaining being a tough skill.

It is seductive to want things to be elegant. It is costly to wait for the elegant solution.

Re: NeuroEvolution – Flappy Bird

#63
post #61
post #15

Earlier quoted context omitted.

It's also using a simple neural network, which is the target of the genetic algorithm, if I understood it right. I haven't seen this combination often - does that make sense in general, or is this just interesting as in playing around with those concepts?

it's a pretty standard procedure to train NNs through GAs but usually not very efficient (e.g. compared to backtracking). in some cases you might lack an easy way to calculate a fitness score out of the NN performance, which is needed to run the GA. i tried training a simple NN with a stupid hill climber some time ago but quickly hit a roadblock even with very few neurons because of local minima ... or maybe bugs. i…

> it's a pretty standard procedure to train NNs through GAs but usually not very efficient (e.g. compared to backtracking).

Different applications, though. Backtracking in the normal sense needs input and expected output (e.g. lots of training data), while GA/EA learns to solve it without explicit wrong/correct actions, just the score at the end.

Re: NeuroEvolution – Flappy Bird

#65
This reminds me of a section (I think it might be the beginning) of "Surely you're Joking, Mr. Feynman" where he recounts fixing radios as a youngster, and the outrage of a client of his that he at times would just sit and stare at the radio. Apparently the guy went around town telling people about this incredible thing he had witnessed -- this kid he knew fixed radios "by thinking."

For every incredibly complicated algorithm designed to solve some really huge problem and deployed at massive scales, there are another thousand little problems at a far lower scale. I did a project at a school once using a GA to automatically create class rosters based on gender balance, student social networks, and grade distribution. This used to be a two-week long guidance counselor project at the school. Now they press a button.

Everyone in this forum clearly understands the impact that software is having on the world. But nevertheless, I still think we underestimate the impact that a more computationally literate society with more readily available computing power can have. I think examples like this excellently written piece of code highlight that.

Here is one -- admittedly very bright -- person who can from the comfort of his browser come up with such a fascinating and ingenious solution to this computational problem. Yes, it is being used for flappy bird, but this would easily extend to maybe a whole set of domain of problems that he/his company/his community could be facing. Yes, it is flappy bird, but this alone could serve as the motivation for an incredible course in a high-school dealing with topics from coding to the theory of evolution. "The Evolution of Cooperation" had a similar effect on me -- Axelrod uses nothing but very elementary Algebra to make such a strong and insightful argument that got me thinking like nothing ever had before.

Here's hoping we make this kind of "play", and this sort of thinking, as widespread as possible in society.

Re: NeuroEvolution – Flappy Bird

#66

At generation 83, a single bird emerged that successfully navigated through several hundred columns (current score 100000+) and showed no signs of failing. It took a few dozen generations to find versions that would make it through a few columns if those columns had gaps without too much vertical distance between them. Somewhere in generation 60-70, I could see versions figuring out how to transition between heights,…

took 230 generations for me to do anything then it just went from dying within the first 3 pipes to mastering it within 4 generations

Re: NeuroEvolution – Flappy Bird

#67

At generation 83, a single bird emerged that successfully navigated through several hundred columns (current score 100000+) and showed no signs of failing. It took a few dozen generations to find versions that would make it through a few columns if those columns had gaps without too much vertical distance between them. Somewhere in generation 60-70, I could see versions figuring out how to transition between heights,…

took 230 generations for me to do anything then it just went from dying within the first 3 pipes to mastering it within 4 generations

On my first run, 2 individuals learnt(?) perfect play in 3 generations.

Re: NeuroEvolution – Flappy Bird

#68

At generation 83, a single bird emerged that successfully navigated through several hundred columns (current score 100000+) and showed no signs of failing. It took a few dozen generations to find versions that would make it through a few columns if those columns had gaps without too much vertical distance between them. Somewhere in generation 60-70, I could see versions figuring out how to transition between heights,…

Generation 10 for me

Re: NeuroEvolution – Flappy Bird

#70
Ahh yes, this is very interesting although I agree with many of the posters here that the model is probably too simplistic even for demo purposes.

For programmers there are even more striking examples of AI out there, complete with code, documentation and explanations.

Check this out:

http://www.primaryobjects.com/2013/01/27/using-artificial-in...

In me this sparked a months long fascination with genetic algorithms solving programming puzzles with a few sets of 'initial memory states' and 'desired memory states'.

The articles start with very simplistic generated programs, and end with a sophisticated model able to generate guess-the-number games using loops, conditionals and handling input from users.

Post reply on HN