Ask HN: Game devs, what is the hardest part about designing an AI for your game?

Like others have said, balancing play-ability is hard. Another problem is users may perceive the AI cheats which hurts fun.

But I'd say another hard thing is that every game needs a different approach to AI. Some need path finding, which is well documented. Others need resource balancing which is less so. The newer and rarer the game mechanic, the more novel the AI.

I would say Machine Learning is not useful except in a handful of games and if yours is not in that category, you risk spending a lot of time for little pay off. And even if in those categories, ML will be working to win not to maximize fun so they could end up being too good or to cheaty looking.

I saw it do it with BA, but I don’t remember the name. It was many years ago. Perhaps RAI? Was that a name?

If your game doesn't give competitors perfect knowledge of the game state, it would make sense for the AI to track previous activity of its current opponent as part of the 'play state' for making decisions. For instance, if it doesn't know where the enemy is, but the enemy has attacked from this direction every time so far, it would prepare for another attack from that direction. Or if the last time the AI made a given offensive move, the player countered effectively a certain way, it may modify its next offensive to overcome that counter. The longer the game, the more it could appear to be learning to counter its opponent, even though, as you say, it's just making the optimal move given the current state of play.

The only Paradox game I've played is Stellaris, which has god-awful AI. Looks like it uses the Clausewitz engine, so perhaps that's the reason. But the AI empires are terrible at balancing their economies, and it doesn't take long for the player to completely outstrip them, even when giving the AIs advanced starts and bonuses. And then the military AI isn't very good, to the point that when it is outmatched by a stronger opponent, it basically sits and waits to be slaughtered.

The only remote challenge comes in the form of the endgame 'crisis' which is basically an empire that appears late in the game with huge armies and has no economy, just scripted reinforcements and possibly some scripted counters from other special AI entities. None of the regular AI empires will have advanced enough to even think about fighting it, so it mostly serves as a check to see if the player has played efficiently enough to that point to stop the threat, and marks a good point to end your game once you have.

The scripted events make for a somewhat interesting game, but if they could provide an AI that actually made the universe feel like a competition in the early and mid game, and like the other empires were a factor in the end game, it would bring the game to another whole level for me. Of course, they'd probably have to fix their general game balance as well, which an a whole other issue.

I think any game with strategy is going to have to use reinforcement learning to have any hopes of being understood by its players. Concerns around cheesing strategies can be mitigated for real time games using constraints and cheesing in turn based strategy games is a symptom that maybe the game you're working on isn't that strategic.

Ultimately Chess, Go, Dota, Starcraft are compelling games to watch and play because the skill cap is so high. No hand coded AI for such a game could ever prepare you to enjoy playing any of those games professionally or get to a point where you can understand professional play. High level strategy in those games would never even be discovered had we not had a large population of players playing those games for long hours.

Odds are most strategy games won't be as successful as any of the above games but the newcomers could still have compelling strategic elements that gamers could explore had they been able to find other players that are slightly better than they are so they learn but at the same time aren't frustrated by too big a skill gap. A small multiplayer community is why the strategy game market is pretty bleak except for a few large players.

Reinforcement learning can provide a better experience to your players because you the game developer don't need be the world expert at the strategies of your game to create a top level AI. You just make sure the rules of the game are there and self play will take care of the rest. You can constrain the AI such that it's enjoyable by both your newest and most devoted players. You can discover OP strategies and better guide your design process. Excel is great for balancing but if you want to make a game complex enough then balancing will be a tremendous challenge without a larger team.

If you're interested in leveraging reinforcement learning for your game, I run yuri.ai which helps game developers do exactly that. So reach out to me at mark@yuri.ai if I can help!