not exactly what you asked for, but if youre looking for a gentle academic introduction to the intersection of ai and games, i recommend [
AI Researchers, Video Games Are Your Friends!](https://arxiv.org/abs/1612.01608)
Ask HN: Building a game for AI Research
31–38 of 38 posts
Re: Ask HN: Building a game for AI Research
#32>"I tried reading TF/PyTorch docs but couldn't find a guide on how to make my game a friendly environment" Are you aware of 'Elf' and 'TorchCraft' by Facebook (creators of Pytorch)? From https://facebook.ai/developers/tools/elf : "ELF provides an end-to-end solution for game research. It includes miniature real-time strategy game environments, concurrent simulation, distributed training over thousands of machines, in…
Thanks for mentioning these!
I started exploring this after seeing TorchCraft on HN yesterday (https://news.ycombinator.com/item?id=16979136) and I worked on a BWAPI AI back in college myself (2009).
I took a look at ELF hoping it was a library or module I could integrate into my project to make it applicable with PyTorch, but it was its own gym complete with full game examples.
Re: Ask HN: Building a game for AI Research
#33-complete game state should be compactly represented, ultimately it will need to be in a form that can be inputted to an NN - player actions should be enumerable (if there are discrete actions) - score is reward, it should be easy to access directly - game will be much easier to learn of there are shorter goals with more score feedback. long puzzles with very sparse reward is still challenging for RL
Funnily enough, our game has these properties as part of its core game design, independent of the AI-friendly aspirations. It's a puzzle-based anti-grav racing game where you try to complete N laps in the fastest time possible. When I was playtesting it I realized "huh there are only 2 buttons involved, the game state exists as a 2D bitmap + a few vectors we can serialize to disk, there are predefined deterministic victory parameters (times for medal) and the randomness is 0 outside of physics. This should be an ideal game for AI training."
While I'm sure I miss some knowledge on optimizing the design, what I'm more interested in learning about are pointers to technical frameworks/API implementation details needed to be friendly to tools like TF/PyTorch for developers to train with the game.
Re: Ask HN: Building a game for AI Research
#34If you're using Unity, I would recommend that you check out Unity Machine-Learning Agents! https://github.com/Unity-Technologies/ml-agents It makes it really easy to make games for reinforcement learning. I worked on it a bit over summer, so if you have any questions, feel free to reach out to me.
We do indeed use Unity! Somehow I was aware of ml-agents, but didn't realize it was a "gymification" library. I thought it was a framework for running stuff within the Unity editor, but seeing that it allows things to be run from a normal python environment, that's awesome. Thanks for making me look at the README and documentation closer.
And here I was thinking I would need to write a native dll wrapper or wait till my next game project to include necessary bits from the outside.
Re: Ask HN: Building a game for AI Research
#35By fun coincidence one of my personal projects has been the opposite; trying to build AIs that can experiment with generalized games. The needs of a project like mine are going to deviate from other suggestions on here because I'm explicitly not working with neural networks or genetic algorithms. The lists other people have put up (esp nrmm) are really good for those, which is not to say mine won't have any overlap.…
Your project sounds exactly like what I envisioned myself dedicating my life to if I went to grad school (ah, another life) instead of starting a company after college. These are all the requirements I try to set for my game designs and gameplay implementations, although I appreciate the details from someone in the weeds and building the project.
What I'm trying to gain more information on though right now are the technical details on how to go about modifying an existing game project to provide this, without building a game from scratch entirely inside of a proprietary AI game-gym framework.
A note on recordable output: It actually runs slower than playing a game because usually you have to do video encoding and file io in realtime which is slower than the RAM -> graphics card -> display pipe.
Re: Ask HN: Building a game for AI Research
#36Something perfect here is OpenAI's Universe [1] enabling you to strap on a uniform AI interface to your project. Here [2] is there systems page for further information/support. [1] https://github.com/openai/universe [2] https://openai.com/systems/
Nice, I didn't know about Universe, I was only familiar with the gym and their DOTA 2 scaffolding (which was a gigantic engineering process, as I learned from https://blog.ycombinator.com/building-dota-bots-that-beat-pr... )
Re: Ask HN: Building a game for AI Research
#37-complete game state should be compactly represented, ultimately it will need to be in a form that can be inputted to an NN - player actions should be enumerable (if there are discrete actions) - score is reward, it should be easy to access directly - game will be much easier to learn of there are shorter goals with more score feedback. long puzzles with very sparse reward is still challenging for RL
Funnily enough, our game has these properties as part of its core game design, independent of the AI-friendly aspirations. It's a puzzle-based anti-grav racing game where you try to complete N laps in the fastest time possible. When I was playtesting it I realized "huh there are only 2 buttons involved, the game state exists as a 2D bitmap + a few vectors we can serialize to disk, there are predefined deterministic v…
Im working on a game RL framework for turn-based games, my aim was to learn about self-play and RL.
I found if you structure the human controller in the exact same way as the AI controllers, you can swap them easily. So I have Agent (abstract), HumanAgent (takes keyboard input), and DqnAgent (DQN learning agent) as the different controllers, the rest of the code is agnostic to the controller. With this setup you can also do things like record your own gameplay.
If your goal is running the track in minimal time, you could reward it at the end (reward = -1* elapsed_time), or as you go (reward=current_speed), or once each lap, etc. These sound similar but may have different training properties. So maybe plan to explore your reward shaping space a bit.
Re: Ask HN: Building a game for AI Research
#38From my viewpoint, as both a researcher and someone who has built frameworks around environments/games: - Each step within the game has to be extremely fast. I.e the game should be able to be run as fast as the machine allows while keeping physics etc. consistent. - Runnable via library import such that there is no drawing to the screen. - Should be easy to reset the environment to an initial state. - RNG state shoul…
> - I highly recommend supporting an identical interface found in OpenAI's gym. Check their docs out. Even better would be to have your game importable as an environment in gym.
> - Configurable screen resolution would be great (eg. output 120x100)
I think both of these things assume you are going to be doing RL from pixels. I think to support a wider variety of RL/control research, you should be able to get the game state in a structured form and not just a flat vector the way gym does it.
But even then, that's still just one branch of AI research. I've seen people optimize how games behave to optimize engagement with the game, and in that setting just controlling the player is not enough. The work I saw looked at controlling level progression to increase engagement, but you could imagine controlling other bits of the game, particularly relevant if your game is not symmetric and the metric you care about is not just making the best AI.
Maybe not AI, but people also do research on how to replace components of games with ML components and the results can be pretty cool, e.g. https://www.youtube.com/watch?v=Ul0Gilv5wvY
Which is just to say that there is not one size fits all approach here.