Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

NeuroEvolution – Flappy Bird

xviniette.github.io

Post reply on HN
Print view View original

41–50 of 103 posts

Re: NeuroEvolution – Flappy Bird

#41

Post by fenomas »

maaaats wrote: post #31
fenomas wrote: post #26

Earlier quoted context omitted.

Wow, that's really unexpected. This isn't my field, but given the simplicity of the inputs and network, and the way commenters are seeing the demo achieve perfect play after anything between 2 and 200 generations, it makes me wonder if this isn't more of a brute-force search than actual learning? That is, it smells like there's a "correct" set of neuron values - where any genome within some tolerance of those values…

"All" evolutionary algorithms are basically a local search with smart heuristics. Where a local search is a brute-force where you move in small directions based on feedback on where you are in the solution space.

I understand how the algorithms work. What I'm suggesting is that the demo seems to behave like a hill-climbing algorithm that's been unleashed on a terrain that's flat everywhere except the solution.

Re: NeuroEvolution – Flappy Bird

#43

Post by w23j »

NicoJuicy wrote: post #42

Where can i learn more about NeuroEvolution or similar algorithms? It's amazing this fits in thos LoC, but i don't see the total picture.. Any books you can recommend?

The big pictures in this case is:

You use a neuronal network to decide at each step to flap the birds wings or not. That is the only output. The input is only the birds height (y-position) and the height of the next hole. (https://en.wikipedia.org/wiki/Feedforward_neural_network)

The question now is how to find the correct weights for the net.

That net is not trained in a traditional supervised way like with gradient descent (which would be more complicated). Instead it uses a genetic algorithms to find new weights for neuronal networks of future generations. (https://en.wikipedia.org/wiki/Genetic_algorithm)

And that's basically it in this case.

Re: NeuroEvolution – Flappy Bird

#45

Post by _kush »

m3adow wrote: post #6

When I first watched it, nothing changed for about 200 generations (thanks for the x5 speed up). I wanted to comment on that, but decided to reload first. Now my 12th generation is flapping for a good two minutes while im writing this comment. It seems only the second time there was machine learning involved.

On generation 7, I got 3 perfect birds http://kush.im/ZeHe

Re: NeuroEvolution – Flappy Bird

#46

Post by raverbashing »

maaaats wrote: post #31
fenomas wrote: post #26

Earlier quoted context omitted.

Wow, that's really unexpected. This isn't my field, but given the simplicity of the inputs and network, and the way commenters are seeing the demo achieve perfect play after anything between 2 and 200 generations, it makes me wonder if this isn't more of a brute-force search than actual learning? That is, it smells like there's a "correct" set of neuron values - where any genome within some tolerance of those values…

"All" evolutionary algorithms are basically a local search with smart heuristics. Where a local search is a brute-force where you move in small directions based on feedback on where you are in the solution space.

Not really, if you add new individuals from random then you're doing some global search (or just have a higher mutation rate - but that has some problems)

Re: NeuroEvolution – Flappy Bird

#47

Post by maaaats »

fenomas wrote: post #41
maaaats wrote: post #31

Earlier quoted context omitted.

"All" evolutionary algorithms are basically a local search with smart heuristics. Where a local search is a brute-force where you move in small directions based on feedback on where you are in the solution space.

I understand how the algorithms work. What I'm suggesting is that the demo seems to behave like a hill-climbing algorithm that's been unleashed on a terrain that's flat everywhere except the solution.

Ah, I see what you mean now. Yeah, the problem space seems a bit simple. I never see any "learning" before it suddenly achieves perfect play.

Re: NeuroEvolution – Flappy Bird

#48

Post by chlee »

NicoJuicy wrote: post #42

Where can i learn more about NeuroEvolution or similar algorithms? It's amazing this fits in thos LoC, but i don't see the total picture.. Any books you can recommend?

Ken Stanley came up with the idea of NeuroEvolution while he was a phd student at the Neural Networks lab at UT-Austin.

You can see some of the older NeuroEvolution papers here: http://nn.cs.utexas.edu/project-list.php

I believe Ken Stanley has gone to become a professor at UCF to continue his research: http://www.cs.ucf.edu/~kstanley/

Re: NeuroEvolution – Flappy Bird

#49

Post by smoyer »

simcop2387 wrote: post #38
JoshTriplett wrote: post #7

At generation 83, a single bird emerged that successfully navigated through several hundred columns (current score 100000+) and showed no signs of failing. It took a few dozen generations to find versions that would make it through a few columns if those columns had gaps without too much vertical distance between them. Somewhere in generation 60-70, I could see versions figuring out how to transition between heights,…

What's interesting is I've run it several times now, and I've gotten the perfect bird about half the time on an early generation (<10), but sometimes it just doesn't want to find it for a while. I'm wondering if the net is just at the threshold of being the minimal set of neurons that are needed for it to be able to get it so easily by random chance.

I was surprised to get a winner on generation 7 the first time I started the trainer. Subsequent trials were much worse.
Post reply on HN