I would love advice from anyone who has experience building API layers for video games for AI (BWAPI, OpenAI's DOTA team, deepmind SC2 AI, etc) for pointers.
Ask HN: Building a game for AI Research
1–10 of 38 posts
Re: Ask HN: Building a game for AI Research
#2OpenAI Gym has a stable API for several video games. If you copy the API, researchers will be able to compare algorithms directly across games which is valuable.
Re: Ask HN: Building a game for AI Research
#3- Each step within the game has to be extremely fast. I.e the game should be able to be run as fast as the machine allows while keeping physics etc. consistent.
- Runnable via library import such that there is no drawing to the screen.
- Should be easy to reset the environment to an initial state.
- RNG state should be seedable.
- I highly recommend supporting an identical interface found in OpenAI's gym. Check their docs out. Even better would be to have your game importable as an environment in gym.
- Configurable screen resolution would be great (eg. output 120x100)
- The environment is "hackable" eg. the maps or levels can be modified or loaded say via some ascii map.
- Should support multiple copies of the game running at once.
- A nice to have would be if the current environment state could be exported and loaded later.
- Expose some information/signals such that a reward signal can be created. Or better yet you define one as the game creator.
Re: Ask HN: Building a game for AI Research
#4Re: Ask HN: Building a game for AI Research
#5- player actions should be enumerable (if there are discrete actions)
- score is reward, it should be easy to access directly
- game will be much easier to learn of there are shorter goals with more score feedback. long puzzles with very sparse reward is still challenging for RL
Re: Ask HN: Building a game for AI Research
#6Re: Ask HN: Building a game for AI Research
#7It has to be fast. Reinforcement learning takes 1000s of hours of game play for each experiment, and you want to do multiple experiments per day on a small cluster. The Atari games in OpenAI Gym run about 200x real-time per core, and the SNES games maybe 20x. Aim for something in that range. 1x is useless for research. OpenAI Gym has a stable API for several video games. If you copy the API, researchers will be able…
They had to do a few hacks to get it working: https://blog.openai.com/more-on-dota-2/?!
The first step in the project was figuring out how to run Dota 2 in the cloud on a physical GPU. The game gave an obscure error message on GPU cloud instances. But when starting it on Greg’s personal GPU desktop (which is the desktop brought onstage during the show), we noticed that Dota booted when the monitor was plugged in, but gave the same error message when unplugged. So we configured our cloud GPU instances to pretend there was a physical monitor attached.
Dota didn’t support custom dedicated servers at the time, meaning that running scalably and without a GPU was possible only with very slow software rendering. We then created a shim to stub out most OpenGL calls, except the ones needed to boot.
Still, Dota uses simplified collision detection. It would be interesting to know how fast an actual physics simulation could run in headless mode.
Re: Ask HN: Building a game for AI Research
#8The needs of a project like mine are going to deviate from other suggestions on here because I'm explicitly not working with neural networks or genetic algorithms. The lists other people have put up (esp nrmm) are really good for those, which is not to say mine won't have any overlap.
Given the choice between supporting something on my list or someone else's, you should definitely prioritize someone else's. If you want a lot of people to do research with your game, optimize for ML.
That being said, because I'm greedy, stuff that matters specifically to me, in order of importance:
- Encapsulated State: I want to be able to get raw access to the game's internal memory, esp through an API. Byte array is fine, labeled is also good. Basically any variables or memory that are used in game logic should be exposed. If possible, this state should be separated from any state information you're using for graphics, sound, etc...
Note that this does not mean you should encapsulate your entire game into 20 or 30 basic variables and just expose that. That's cheating. Classification of a game state is as important as building a strategy, so the point is to provide your state at a (relatively) low level with only a few abstractions, or at least abstractions that can easily be stripped out for training.
- Headless Mode: I want to be able to run the game on a server or in a low resource environment by disabling as much of the graphics stack as possible. Linux support is also obviously useful since the majority of the server world is using it.
- API-driven input: I want to be able to control the game without mocking button presses on a keyboard. Extra plus if your game is controllable via a scripting language like Javascript, it's trivial for someone to build their own adapter if they have something to call into.
- Serializing/loading state: Emulators are great for AI research because you can export out the entire state of the game and reload it later for testing/comparing results. This isn't essential, just really really useful.
- Easily recordable output: This is a tiny thing that just makes life way easier. If you can pipe your game's output to a video stream without recording it off a desktop (ie, can I stream your game from a headless server), then it just makes it a lot more attractive for building demos or toys.
Loads of bonus points if your graphical display can be run as a separate process from your game logic. More bonus points if they can communicate over a network. All of the bonus points if your graphics engine is embeddable in a web-browser or is similarly scriptable to your game logic.
And some stuff I don't really care about:
- Synchronous game logic: Most games try to keep their logic divided into synchronous chunks that are executed every frame. This is not particularly important for the types of research that I'm doing, async code is fine. Your game will probably be running on a remote server anyway, so someone in my position is hardly ever going to be looking at every single frame in series.
- Seedable RNG: It's nice. I guess. But unless someone is running your game locally and interacting with it via synchronous steps, they're going to be dropping random frames anyway. So it doesn't really matter.
- Extreme optimizations: I'm overstating this a bit - obviously your game needs to be optimized. If your game requires a GPU to run at a decent clip, that's a problem. BUT, if your game is fast enough to run reasonably well on limited hardware (especially in a streaming/headless mode), that's good enough. Aim to support single-core hardware and small dedicated devices like the Raspberry Pi.
This is terrible advice if you want to support reinforcement learning. In all likelihood you should optimize the crap out of your game for that. But, for the type of research I'm doing, I will practically never be running a game at more than 1-2x speed.
Portability and elegance > raw speed, unless you care about training a genetic algorithm or something.
- A fitness function: Honestly, probably not even the traditional ML crowd needs one. If you expose your game's state in a way that they can consume, then anyone can write their own fitness function on top of it. You don't hurt anything by adding one though, I guess.
Re: Ask HN: Building a game for AI Research
#9However, from what I understand, this limits your game to a simple Markov Process, which not all games likely expressed like that. (In my case I can only use this approach if I don't allow neural networks to have memory between frames)
Another benefit is being able to debug in real-time how decisions are being calculated via the model, like a neural network.
Re: Ask HN: Building a game for AI Research
#10From my viewpoint, as both a researcher and someone who has built frameworks around environments/games: - Each step within the game has to be extremely fast. I.e the game should be able to be run as fast as the machine allows while keeping physics etc. consistent. - Runnable via library import such that there is no drawing to the screen. - Should be easy to reset the environment to an initial state. - RNG state shoul…