Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning Machine Learning for Game Development

ai.googleblog.com

Post reply on HN
Print view View original

31–40 of 69 posts

Re: Learning Machine Learning for Game Development

#31

Post by summerlight »

Jasper_ wrote: post #16
Dzugaru wrote: post #6

Earlier quoted context omitted.

Anecdotally: I've never seen a "simple state machine" or "simple behaviour tree" in game AI. We've recently started using deep reinforcement learning for our games and its almost like a miracle how simple, effective and scalable the system is. There are some mentioned problems like designing rewards for player enjoyment, but its definitely got a massive reduction in engineering effort.

> Anecdotally: I've never seen a "simple state machine" or "simple behaviour tree" in game AI. That's effectively what game AIs are, today. Users want an AI they can model and simulate in their head, and isn't too brutal of a challenge. Today's machine learning cannot provide a model like that.

The whole point of this research is to early study on unknown aspects of game's design and its consequence on user behaviors and game balance. AI should be able to find and exhibit game play unexpected to the designer, which is not easily achievable with a tight control given to game designers. It's more of systematic state space exploration, not making AI fun to play with.

Re: Learning Machine Learning for Game Development

#32

Post by vladTheInhaler »

Quarrelsome wrote: post #22
CJefferson wrote: post #3

I'm in this research area, it's facinating, but also (like many things in AI) easy to get wrong. The trickiest bit is deciding exactly what you want your AI to optimise for -- If you want a game where "anyone can win", you end up with coin toss. If you want a game where "skill is everything", you can accidentally end up with "who can click fastest". On a project I recently worked on (and I'm writing up), we ended up…

I feel like some of these questions are a bit backwards. Isn't it better to give the model an arena to play within and try to make all of the options _viable_? I.E. Balance is maintained when the average of the weapons the gladiators choose to take into the arena are diverse and have less bearing on the win outcome? This means that rather focusing on what is merely broken one is focusing on both ends, what is broken…

Attempting to optimize the card generation process, or deck building, or just play itself seem like unbelievably difficult problems. Some effort at least has been made in the first part (see RoboRosewater) but the other two seem almost intractable, given the combinatorial explosion of possibilities in deckbuilding, the incredibly nonlinear interactions between cards (Splinter Twin is a 1/10 card, Pestermite is maybe 3/10, Splinter Twin + Pestermite literally wins the game on the spot).

Re: Learning Machine Learning for Game Development

#33

Post by jonbaer »

adkadskhj wrote: post #11
CJefferson wrote: post #3

I'm in this research area, it's facinating, but also (like many things in AI) easy to get wrong. The trickiest bit is deciding exactly what you want your AI to optimise for -- If you want a game where "anyone can win", you end up with coin toss. If you want a game where "skill is everything", you can accidentally end up with "who can click fastest". On a project I recently worked on (and I'm writing up), we ended up…

Personally, i'd like to see AI used for "simple" things, not often tied to the core gameplay loop itself. Ie conceptually if you could tie the right gameplay components to an RNG function then the user could get new/unexpected experiences for a longer time. The problem is RNG often makes crap outputs. I'd be curious to see ML take this problem. I imagine the examples are limitless, but one that comes to mind might be…

I think you are right. There are SO many small mini games inside of nearly all RTS games which AI has yet to actually conquer, I will give another simple example, given a random map (and resources) identify chokepoints. As a human this is a huge clue on how to go about a game strategy (policy) but I have yet to find an AI do this (or optimize that part). From what I gather most time is spent on actually optimizing random build orders and modifying them along the way.

Re: Learning Machine Learning for Game Development

#34

Post by sfjailbird »

Synaesthesia wrote: post #30
sfjailbird wrote: post #18

The most obvious use of ML in this context would be to predict player actions in multiplayer games. Most such games extrapolate the actions of remote players in order to give the illusion that there is no lag between updates. I bet for many situations, a good ML implementation could make this work almost flawlessly. No idea if anyone is doing this yet. Also, the headline is wrong: It's 'Leveraging Machine Learning fo…

So how will it be able to predict if I will turn left or right next?

With a training set of all previous sessions played, I bet this would often be easy to predict.

Re: Learning Machine Learning for Game Development

#37

Post by YeGoblynQueenne »

CJefferson wrote: post #3

I'm in this research area, it's facinating, but also (like many things in AI) easy to get wrong. The trickiest bit is deciding exactly what you want your AI to optimise for -- If you want a game where "anyone can win", you end up with coin toss. If you want a game where "skill is everything", you can accidentally end up with "who can click fastest". On a project I recently worked on (and I'm writing up), we ended up…

>> The trickiest bit is deciding exactly what you want your AI to optimise for -- If you want a game where "anyone can win", you end up with coin toss. If you want a game where "skill is everything", you can accidentally end up with "who can click fastest".

In Magic: the Gathering this balancing act is achieved by a set of constantly changing "environments" (sets of cards) that are updated every few months, when a new set of cards is published. Sets "rotate" in and out of the various environments and there's a few where it's legal to play with all cards ever printed (excluding some that are banned or restricted for being OP- Over Powered).

When a new set rotates (its cards become legal) in a given environment, there will be a reshuffle of the balance of power between already-established decks, that now acquire new cards or lose cards they needed, and some new strategies become available resulting in new decks being designed. Eventually the dust settles and the "tier one" decks (the ones that win the most) are found. Then tournament play in particular becomes a rock-paper-scissors affair, until the next update.

In some of the environments were all cards from all sets are legal (again, minus the ones in the banned or restricted lists) games indeed often come down to a coin toss: the player who goes first wins (in one turn, through some OP combo, typically).

I suspect that a perfectly balanced game is impossible to have and would probably be boring even if it was possible to have it. An element of creativity with constantly updated design parts, like in M:tG is probably the best one can do.

Re: Learning Machine Learning for Game Development

#38

Post by CJefferson »

ericjang wrote: post #10
CJefferson wrote: post #3

I'm in this research area, it's facinating, but also (like many things in AI) easy to get wrong. The trickiest bit is deciding exactly what you want your AI to optimise for -- If you want a game where "anyone can win", you end up with coin toss. If you want a game where "skill is everything", you can accidentally end up with "who can click fastest". On a project I recently worked on (and I'm writing up), we ended up…

Interesting! Is there some kind of meta-objective for "AI optimal" that could have replaced the 6 or 7 iterations you did with human R&D? For instance, if you had real human playtesters interacting with the prototypes, is there some signal you could extract to measure that it's "good"?

The problem is AIs are very good at optimising what you asked them for, rather than what you meant to ask for, and figuring out what you want is super hard :)

As a simple example:

* Start by optimising "players can always do something on their turn" -- but that just ends up with everyone always having exactly one thing they can do (no choice).

* So then say "give players more things to do each turn" -- but then they end up being able to do everything every turn (the game gives them too much 'money' (still not really a choice)

* OK, so we want to force players to make a choice -- so we say "No, give players as much choice as possible, but make sure if they choose an option it blocks off others (in practice, make as many sets of maximal tasks as possible)" -- but then the AI will make sure every turn every player can do (for example) exactly 3 out of 6 things (any 3), and make sure no matter how well or badly they play they still always get to choose 3 from 6, so the game doesn't really progress, or vary.

So, what we want is choice, but also variability, and progress, and players to feel like they are effecting the game, but also don't let one player run away too early, but also don't make it just "feel random who wins", etc.

Re: Learning Machine Learning for Game Development

#39

Post by vvanders »

YeGoblynQueenne wrote: post #37
CJefferson wrote: post #3

I'm in this research area, it's facinating, but also (like many things in AI) easy to get wrong. The trickiest bit is deciding exactly what you want your AI to optimise for -- If you want a game where "anyone can win", you end up with coin toss. If you want a game where "skill is everything", you can accidentally end up with "who can click fastest". On a project I recently worked on (and I'm writing up), we ended up…

>> The trickiest bit is deciding exactly what you want your AI to optimise for -- If you want a game where "anyone can win", you end up with coin toss. If you want a game where "skill is everything", you can accidentally end up with "who can click fastest". In Magic: the Gathering this balancing act is achieved by a set of constantly changing "environments" (sets of cards) that are updated every few months, when a ne…

Perfectly balanced games are plentiful, take rock, paper, scissors as an example.

What makes those games interesting is the concept of Yomi[1] and learning to read an opponent. There's a decent amount of literature in the fighting game space and a bunch of other genres overlap as well.

I remain pretty skeptical of ML being a primary tool here since a lot of Yomi is psychological and not necessarily an optimization problem.

[1] https://www.gamasutra.com/blogs/CalebCompton/20190211/336365...

Post reply on HN