Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Learning Machine Learning for Game Development

ai.googleblog.com

Post reply on HN
Print view View original

51–60 of 69 posts

Re: Learning Machine Learning for Game Development

#51

Post by ZephyrBlu »

TylerLives wrote: post #47
ZephyrBlu wrote: post #44

I have a bone to pick with these "win probability" charts. If it was truly a "win probability" chart, that means it's a forecast. Except it's a shit forecast because you're trying to predict really freaking far into the future (You don't even know how far because the game could end at any point in time). It also makes zero sense. Think about it, I say "you have a 30% chance of winning the game from this position". Wh…

> Think about it, I say "you have a 30% chance of winning the game from this position". What does that even mean? If I play well I have a 30% chance of winning? If my opponent is of equal skill I have a 30% chance of winning? It's completely uninterpretable. It means that the AI that generated the data has 30% chance of winning, or at least that's what it's trying to estimate.

That doesn't make sense in the context of a game though. You can't freeze the game and say "I have a xx% chance to win here", because the state of the game continues evolving over time.

The generated value is based on a snapshot of the game and isn't related to the final outcome at all, so I believe saying it's a "win probability" is completely wrong.

Re: Learning Machine Learning for Game Development

#52

Post by jonbaer »

loosetypes wrote: post #43
jonbaer wrote: post #33

Earlier quoted context omitted.

I think you are right. There are SO many small mini games inside of nearly all RTS games which AI has yet to actually conquer, I will give another simple example, given a random map (and resources) identify chokepoints. As a human this is a huge clue on how to go about a game strategy (policy) but I have yet to find an AI do this (or optimize that part). From what I gather most time is spent on actually optimizing ra…

+1 among my favorite aspects in certain rts games. My favorite case is empire earth, where there’s randomness in resource clustering as well as natural terrain. Depending on your strategy/units (and your opponents’) it’s increasingly grey whether a locale is even a choke point. The homogeneity of, for example, Starcraft maps and limited unit options always made me feel like we were playing on an excel spreadsheet.

This sounds very familiar to the game I play 0ad (https://play0ad.com/), you have to actually find an enjoyable way to play RTS games but in this game I pick myself vs. 7 AI with a random map, you never get the same allies and how you develop a strategy around the market (ie. chokepoints) is a good 20% of the game. I just wish the AI decisions on diplomacy weren't so random but it sounds like that mini game will soon be solved, https://spectrum.ieee.org/tech-talk/robotics/artificial-inte...

Re: Learning Machine Learning for Game Development

#53

Post by follower »

Just recently watched "Deep Learning: Beyond the Hype" which gave some interesting insights into related experiences from a team at EA:

https://www.gdcvault.com/play/1025098/Deep-Learning-Beyond-t...

https://www.youtube.com/watch?v=yA-lJy52Ais

From the Game Developers Conference (GDC 2018) the talk covers a bit of the story of building on the work of a NN playing Atari-era games in order to enable playing a modern AAA FPS (First Person Shooter).

(A text PR article interview of the speaker can be found here: https://www.ea.com/en-gb/news/teaching-ai-agents-battlefield... )

While trying to find the video again I also found this later talk "AI for Testing The Development of Bots that Play 'Battlefield V'":

https://www.gdcvault.com/play/1026308/AI-for-Testing-The-Dev...

https://www.youtube.com/watch?v=_cslewPyKks

Re: Learning Machine Learning for Game Development

#54

Post by anonymoushn »

ZephyrBlu wrote: post #50
jwilber wrote: post #49

Earlier quoted context omitted.

Disagree with basically everything written here. The neural net is a well-researched cnn variant - the activations are quite interpretable these days. The metric’s interpretability is independent from the neural net used for gameplay, so the black box comment doesn’t make sense (unless I’m misunderstanding the comment...). The metric is also very human friendly: think of it as 30% of players in the given position, at…

> The metric is also very human friendly: think of it as 30% of players in the given position, at the given game’s stage and state, successfully continued on to a victory What I am trying to say is that this is a meaningless piece of information, because that's not how games work. You can't freeze the game state and say "I have a xx% chance to win here", because the state of the game continues evolving over time. If…

What's "the context of the game" and why can't it be included in the encoding of the game state?

Re: Learning Machine Learning for Game Development

#55

Post by anonymoushn »

ZephyrBlu wrote: post #51
TylerLives wrote: post #47

Earlier quoted context omitted.

> Think about it, I say "you have a 30% chance of winning the game from this position". What does that even mean? If I play well I have a 30% chance of winning? If my opponent is of equal skill I have a 30% chance of winning? It's completely uninterpretable. It means that the AI that generated the data has 30% chance of winning, or at least that's what it's trying to estimate.

That doesn't make sense in the context of a game though. You can't freeze the game and say "I have a xx% chance to win here", because the state of the game continues evolving over time. The generated value is based on a snapshot of the game and isn't related to the final outcome at all, so I believe saying it's a "win probability" is completely wrong.

If I have pocket aces and you have 7-2 off-suit, does a win probability exist for my aces against your 7-2?

If Ryu is knocked down and he has to dragon punch to beat a meaty command grab and has to throw to beat an opponet who is attempting to block his dragon punch, and both players are at very low health, and these two options dominate all other options, such that the entire game hinges on a single round of a double-blind guessing game with only two possibilities, does a win probability exist for Ryu?

Re: Learning Machine Learning for Game Development

#56

Post by ZephyrBlu »

anonymoushn wrote: post #55
ZephyrBlu wrote: post #51

Earlier quoted context omitted.

That doesn't make sense in the context of a game though. You can't freeze the game and say "I have a xx% chance to win here", because the state of the game continues evolving over time. The generated value is based on a snapshot of the game and isn't related to the final outcome at all, so I believe saying it's a "win probability" is completely wrong.

If I have pocket aces and you have 7-2 off-suit, does a win probability exist for my aces against your 7-2? If Ryu is knocked down and he has to dragon punch to beat a meaty command grab and has to throw to beat an opponet who is attempting to block his dragon punch, and both players are at very low health, and these two options dominate all other options, such that the entire game hinges on a single round of a doubl…

You have created 2 scenarios with very small actions spaces. Most games have enormous action spaces, so you can't easily compute a probability.

Take Chess for example. Chess engines evaluate millions of scenarios for every move. They don't generate a "win probability" value though, they evaluate board positions.

Re: Learning Machine Learning for Game Development

#57

Post by ZephyrBlu »

anonymoushn wrote: post #54
ZephyrBlu wrote: post #50

Earlier quoted context omitted.

> The metric is also very human friendly: think of it as 30% of players in the given position, at the given game’s stage and state, successfully continued on to a victory What I am trying to say is that this is a meaningless piece of information, because that's not how games work. You can't freeze the game state and say "I have a xx% chance to win here", because the state of the game continues evolving over time. If…

What's "the context of the game" and why can't it be included in the encoding of the game state?

Context of the game = previous events that occurred.

Don't you think it's a bit odd to be forecasting a "win probability" value without taking the events of the game into account?

> why can't it be included in the encoding of the game state?

How do you encode time series or event data into singular values?

Re: Learning Machine Learning for Game Development

#58

Post by anonymoushn »

ZephyrBlu wrote: post #56
anonymoushn wrote: post #55

Earlier quoted context omitted.

If I have pocket aces and you have 7-2 off-suit, does a win probability exist for my aces against your 7-2? If Ryu is knocked down and he has to dragon punch to beat a meaty command grab and has to throw to beat an opponet who is attempting to block his dragon punch, and both players are at very low health, and these two options dominate all other options, such that the entire game hinges on a single round of a doubl…

You have created 2 scenarios with very small actions spaces. Most games have enormous action spaces, so you can't easily compute a probability. Take Chess for example. Chess engines evaluate millions of scenarios for every move. They don't generate a "win probability" value though, they evaluate board positions.

> Most games have enormous action spaces, so you can't easily compute a probability.

Saying you can't compute something by hand on a single sheet of paper is different from saying it doesn't exist.

In Chess, your win probability is either 0 or 1 (you either can force a win or you cannot), but most chess engines have a real-valued estimate of their equity in the game that is between 0 and 1.

Re: Learning Machine Learning for Game Development

#59

Post by anonymoushn »

ZephyrBlu wrote: post #57
anonymoushn wrote: post #54

Earlier quoted context omitted.

What's "the context of the game" and why can't it be included in the encoding of the game state?

Context of the game = previous events that occurred. Don't you think it's a bit odd to be forecasting a "win probability" value without taking the events of the game into account? > why can't it be included in the encoding of the game state? How do you encode time series or event data into singular values?

> Don't you think it's a bit odd to be forecasting a "win probability" value without taking the events of the game into account?

No. To the extent that these events have no impact on the rules or the outcome of the game beyond the current game state, they don't influence how much equity a player has in the game. You may be interested in "The probability that I win this game given my beliefs about the tendencies of my opponent and my own tendencies, not given optimal play." That's a fine thing to be interested in, but there are other things that can reasonably be called "win probability."

> How do you encode time series or event data into singular values?

I don't know, how do you encode a sequence of 1-byte values into a 1kb string of text? You can read the alphazero paper if you want to learn how a sequence of go game states were encoded.

Re: Learning Machine Learning for Game Development

#60

Post by ZephyrBlu »

anonymoushn wrote: post #58
ZephyrBlu wrote: post #56

Earlier quoted context omitted.

You have created 2 scenarios with very small actions spaces. Most games have enormous action spaces, so you can't easily compute a probability. Take Chess for example. Chess engines evaluate millions of scenarios for every move. They don't generate a "win probability" value though, they evaluate board positions.

> Most games have enormous action spaces, so you can't easily compute a probability. Saying you can't compute something by hand on a single sheet of paper is different from saying it doesn't exist. In Chess, your win probability is either 0 or 1 (you either can force a win or you cannot), but most chess engines have a real-valued estimate of their equity in the game that is between 0 and 1.

> but most chess engines have a real-valued estimate of their equity in the game that is between 0 and 1

My understanding is that Chess engines do not evaluate a 0-1 value to representing chance to win.

They evaluate the perceived material advantage from the current board position.

See: https://www.chess.com/blog/ChessMN16/how-to-read-engine-eval...

Post reply on HN