AI Programmer’s Bookshelf

What you describe in your football example is often called "Behavior Tree", and there are ways to learn those using machine learning (usually by reinforcement learning, using neural networks and/or genetic programming). There was a popular video showing this applied on Mario [1] ... and a paper published 7 years before that video doing the same thing [2] (more context about this in [3]). I remember seeing something on Gamasutra saying that similar methods are used in some AAA games.

[1] https://www.youtube.com/watch?v=qv6UVOQ0F44 [2] http://julian.togelius.com/Togelius2009Super.pdf [3] http://togelius.blogspot.co.uk/2016/04/the-differences-betwe...

That said, the references for AI and machine learning are quite old. Particularly the machine learning parts. The only ML texts on the list are Mitchell and Duda and Hart. The former is extremely outdated at this point. That's not Mitchell's fault -- it was a nice book for learning the basics of machine learning in 1997 when it was published, but all the developments that have made ML a hot subject have occurred since then and in areas that the book simply didn't predict coming. Duda and Hart, similarly, was the bible for certain subfields of ML for a long time, but it won't tell you what everyone's been doing in the past 15 years when ML exploded onto the wider scene.

If I were to add one book, it would be Kevin Murphy's excellent text (https://www.amazon.com/Machine-Learning-Probabilistic-Perspe...). There's no one book that will give you a complete picture of the field, but his is I think the closest available and does a solid job of preparing you with enough fundamentals that you can extend your knowledge from there on your own.

And for those who are reading this, and are interested in game AI with interest in making true machine learning integrated into their characters, I gope this will help keep you inspired.

So, like you, I had grandeous ideas and vision about adaptive, machine learning approach to controlling my character in a platform-based fighting game. They would learn from previous mistakes and improve themselves, to create a truly interactive AI, one that can challenge the player beyond just memorizing his pre-programmed state machine patterns or giving him inhuman reaction times. Then I ran into the same issues, the game must run at 60 fps, and constant learning cannot be done, so i've implemented a basic AI, but i know every way it will act, nothing is amazing.

Until one day I went back and looked at my AI approach, I realize I could still use machine learning, but perhaps use a smaller neural network, and a different learning algorithm. So, after implementing a combination of reenforcement learning and evolutionary learning. I let them train for a day. Then something amazing happened, and it 's what i imagine a parent feels when their kids learns to do something: it saved itself. Initially, starting out the AI would just spam buttons and usually end up jumping off the platform and killing itself, or stray away from the edge and just not touching the control stick, but this time, he got knocked off and he saved himself.

It was an amazing feeling, I never taught it to do that, but I gave it the ability to learn to do that, and that was extremely liberating.

So i encourage people to not give up on the ML AI for videogames, I know deep mind recently teamed up with blizzard to make a StarCraft2 AI, and that looks awesome.