Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Can a Machine Beat Mario but Not Pokemon?

medium.com

Post reply on HN
Print view View original

61–70 of 76 posts

Re: Why Can a Machine Beat Mario but Not Pokemon?

#61

Post by VLM »

dnautics wrote: post #53
VLM wrote: post #49

Earlier quoted context omitted.

Any multi player game with hidden data lets you troll the other players. So if its the dealer and two or more players sharing a deck you can mess with them. Everyone counts cards so under extremely rare circumstances given card count X you can play as if the count is X+1 or X-1 thus mess with the minds of people who accurately believe the count to be X, possibly leading to an advantage, if, statistically, they're in…

What's the hidden data in blackjack? All the cards are face up except the one dealer card.

New player : Whats card counting?

Intermediate player (where its possible to mess with people) : I'm not good at this, whats the count now?

(edited to note: I think external card counting devices like a phone app are illegal everywhere, which does leave some of the fun in the game)

Experienced player : Everyone is effectively perfect at card counting making it boring again.

Something I don't like about blackjack as a game is casinos minimize the entertainment of the game by having like six decks that are shuffled constantly, how super boring why not just play slots or set your cash on fire and watch it burn. If you play it at home with one deck multiplayer and no shuffling foolishness its a moderately entertaining arithmetic game with multiple meta levels of "press your luck" although the house doesn't have guaranteed boring income under that format, which is why casinos won't play that way. Its a game that's too much fun not to be made boring, or something like that. Casinos could take sex and make it boring. Not a fan.

Re: Why Can a Machine Beat Mario but Not Pokemon?

#62

Post by tveita »

phire wrote: post #5

The complexity of Pokemon is much lower than the author implies. It's not really an open world game. There are lots of choke points where your branching factor is limited drops down to two, continue forward to the next trainer or go back and regroup. There is no issue regrouping in Pokemon, you always end up stronger. There is also very little punishment for pushing fowards and failing, you lose half your money and m…

The article is a bit misleading, of course you can make a bot that can beat Pokemon, just like you can make bots that can play e.g. World of Warcraft, but you'd end up embedding knowledge of concepts like trainers, shops, battles and probably even map-specific info into the code.

What we probably can't do at the moment is make a bot that can learn to play any Pokemon-like game, with no specific knowledge of the mechanics, just by operating the controls and observing the pixels.

For that you need to try a large number of strategies and see which works better, and "very little punishment" is a disadvantage when you're trying to tell a good strategy from a bad one. You'd much prefer being swiftly and brutally punished for mistakes so you can go back and try something different.

E.g. how do you know that winning is good and dying is bad? How do you know that getting past a choke point is better than just wandering around town? How do you know getting stronger is progress, but reshuffling your inventory isn't?

Re: Why Can a Machine Beat Mario but Not Pokemon?

#63

Post by MereInterest »

ziaddotcom wrote: post #17
pas wrote: post #14

Earlier quoted context omitted.

Discover as much of the map as possible? Talk to as many NPCs as possible? Try to get as many different dialogues from NPCs as possible? (I assume that to get to the you won dialogue means you encounter more dialogues than if you cleverly lose.)

With the bayesian mesh you could arbitrarily apply heuristics over what otherwise looks pretty similar to a simple coordinate map. If you happen to die near a certain node you could add an ajacent node of "scary" or whatever. Reverse idea for nodes where you get free food and pokemon.

Do you have a link to any papers about bayesian meshes? This sounds like a really neat technique, but my google-fu is failing me.

Re: Why Can a Machine Beat Mario but Not Pokemon?

#64

Post by chongli »

Retric wrote: post #51
chongli wrote: post #46

Earlier quoted context omitted.

The RPG may have been in another language, but it presumably still depicted human or humanoid characters, objects, enemies, etc. Your sister knew far more about life (and consequently the game) than what a computer would know. Just having a basic concept of reality, that the world is made up of objects and agents, that some things can be interacted with, cause and effect, etc. is way beyond what a computer has (which…

This was simple menu driven early Final Fantasy style game on rails. So, some basic path finding and press X near stuff would be useful. But, it did not need to react to enemies, or have complex environments like Zelda games do. On the other hand, the meaning of all those random text blurbs was rather critical.

Without being told, a machine is not even going to know the difference between a menu and the game world. Pathfinding? You need to have a goal in order to do that. How does a machine set high level goals without having any high level concepts of the game itself? It's all just a bunch of memory to the machine. There's no meaning attached to any of it.

Re: Why Can a Machine Beat Mario but Not Pokemon?

#65

Post by Bartweiss »

dragontamer wrote: post #58
nbeleski wrote: post #18

Earlier quoted context omitted.

Given the results obtained by the OpenAI in Dota[1] (with asymmetrical teams nonetheless) I am pretty confident RL could be used to train a pretty efficient pokemon pvp agent. From my experiences the nuances and mindgames/predictions in a pokemon battle are much simpler than those in a high level chess/go game. I would say the model isn't as straightforward as the Mario or Sonic AI players, but is still achievable. A…

DOTA is a bad example. Poker is a better example, because Nash-Equalibrium estimating algorithms have begun to perform better than humans in the past year or two. Pokemon, like Poker, is a game of bluffing and partial information. I expect Pokemon's optimal strategy to be the same mix of fold (aka: switch your Pokemon out to a defensive Pokemon... eating an attack but minimizing the opponent's damage to your team), a…

The poker analogy seems like the right one to use, although Pokemon is made messier by the level of variance. (Meaning both "semi-random effects" and also "far more than 52 possibilities for mon and moves".) I'd imagine the completely-hidden playstyles would be incredibly hard for an AI to learn, but the popular Showdown style that has team preview might be workable. The poker analogy seems like a good one, at least for studying the sorts of things an agent would need to do.

There's definitely a recognizable 'tempo' to pokemon, where A picks a move that threatens B, B switches to something that can take it and threaten back, then A in turn switches to take the hit and threaten back. Which, much like just accurately betting your hand strength in poker, is enough to beat a lot of amateurs. The metaphor goes from there - though I might use 'raise' for leaving a threatened pokemon exposed, which lets us differentiate a strong hand ("I'll use a coverage move with higher speed") from a bluff ("I can hit his switch if I call it.") As an example, opening Koko v Landorus. The fold is switching Koko to Skarmory, the honest raise is HP Ice, and the bluff is Thunderbolt.

The basic ebb and flow of the game seems like it's that and one more layer - double switches and attempts to predict them. Above that, there's just not enough probability mass left to benefit from trying to triple switch, counter-counter-switch, and so on.

Of course, it's all made vastly more complicated by trying to trap, set hazards or status, and make space for setup moves. I'm not sure what it would take to get an unsupervised learner to value e.g. Rocks appropriately. My experience has been that neural nets struggle badly on assessing that sort of long term state change, though of course I'm not working at OpenAI or DeepMind levels.

Re: Why Can a Machine Beat Mario but Not Pokemon?

#66

Post by lacker »

I think this article's key conclusion is incorrect, and that is is likely possible for modern AI techniques to beat Pokemon.

The main reason a machine has not already beaten Pokemon is that it is a nontrivial amount of work to connect a standard AI algorithm to a new video game, and nobody has crossed that hurdle for Pokemon. Mario is one of the most popular video games of all time, and so it is one of the few games that people have connected AI to.

If there was an AI that was connected to a Pokemon game, I am fairly confident the AI would be able to beat Pokemon. The article discusses the problem of having an unclear goal metric. That doesn't seem like a very hard problem to me - you can start with something like, win as many battles as possible. You might beat the game just randomly after you become powerful enough, or you might need some more tweaks to the metrics, but it doesn't seem like that should be a showstopper.

A lot of people are pointing out that Pokemon is fundamentally harder than games like poker or go. That is true, but the bar in this article is beating the game. For poker or go, AI is now better than any human. That is a much higher bar, that is not even relevant for the single-player Pokemon game.

Re: Why Can a Machine Beat Mario but Not Pokemon?

#67

Post by dragontamer »

Bartweiss wrote: post #65
dragontamer wrote: post #58

Earlier quoted context omitted.

DOTA is a bad example. Poker is a better example, because Nash-Equalibrium estimating algorithms have begun to perform better than humans in the past year or two. Pokemon, like Poker, is a game of bluffing and partial information. I expect Pokemon's optimal strategy to be the same mix of fold (aka: switch your Pokemon out to a defensive Pokemon... eating an attack but minimizing the opponent's damage to your team), a…

The poker analogy seems like the right one to use, although Pokemon is made messier by the level of variance. (Meaning both "semi-random effects" and also "far more than 52 possibilities for mon and moves".) I'd imagine the completely-hidden playstyles would be incredibly hard for an AI to learn, but the popular Showdown style that has team preview might be workable. The poker analogy seems like a good one, at least…

> The metaphor goes from there - though I might use 'raise' for leaving a threatened pokemon exposed, which lets us differentiate a strong hand ("I'll use a coverage move with higher speed") from a bluff ("I can hit his switch if I call it.") As an example, opening Koko v Landorus. The fold is switching Koko to Skarmory, the honest raise is HP Ice, and the bluff is Thunderbolt.

I'd argue that the raise is U-Turn :-). Which instant-wins any switching contest (ex: U-Turn on the switch, leaving the option to switch into Magnezone to trap the Skarmory, or if Lando stays in you can switch to your dedicated Lando counter... not that Lando really has a solid counter mind-you, but you get the idea.).

The U-Turn war however, between Lando and Koko demonstrates the bluffing game once again. Koko staying in and doing something weird like Calm Mind, or even Reflect/Light Screen would be absurd, but it would definitely beat the Lando U-Turn in most cases.

Re: Why Can a Machine Beat Mario but Not Pokemon?

#68

Post by [deleted] »

chongli wrote: post #64
Retric wrote: post #51

Earlier quoted context omitted.

This was simple menu driven early Final Fantasy style game on rails. So, some basic path finding and press X near stuff would be useful. But, it did not need to react to enemies, or have complex environments like Zelda games do. On the other hand, the meaning of all those random text blurbs was rather critical.

Without being told, a machine is not even going to know the difference between a menu and the game world. Pathfinding? You need to have a goal in order to do that. How does a machine set high level goals without having any high level concepts of the game itself? It's all just a bunch of memory to the machine. There's no meaning attached to any of it.

[deleted]

Re: Why Can a Machine Beat Mario but Not Pokemon?

#69

Post by Retric »

chongli wrote: post #64
Retric wrote: post #51

Earlier quoted context omitted.

This was simple menu driven early Final Fantasy style game on rails. So, some basic path finding and press X near stuff would be useful. But, it did not need to react to enemies, or have complex environments like Zelda games do. On the other hand, the meaning of all those random text blurbs was rather critical.

Without being told, a machine is not even going to know the difference between a menu and the game world. Pathfinding? You need to have a goal in order to do that. How does a machine set high level goals without having any high level concepts of the game itself? It's all just a bunch of memory to the machine. There's no meaning attached to any of it.

Menus where extremely limited. Repeating (Hello) > (Hello) > (Hello) is little different than just walking into a wall constantly.

Item Shops would IMO be the largest issue, but I did not see anything that looked like one.

Re: Why Can a Machine Beat Mario but Not Pokemon?

#70

Post by setr »

WorkLifeBalance wrote: post #47
setr wrote: post #20

Earlier quoted context omitted.

I think one major difference between pvp games and games like mario/pokemon campaign is that in the former case, the machine is competing with a meat learning algorithm. That is, it doesn’t have to find a good enough solution to a static map, but best an alternative optimization strategy (powered by wetware) before it can reasonably claim success. That, and ofc, games with a competitive scene come about because of th…

Meta doesn't imply cycling, just that the best strategy may be a mixed strategy which involves randomly picking between different pure strategies. Instead of cycling, the meta ought to converge to a Nash equilibrium. With the right mixed strategy, an opponent choosing a pure strategy would be at a disadvantage. Randomising over a huge choice of pure strategies may be infeasible of course and in the real world players…

I wasn’t suggesting that meta’s naturally cycle, but that the ideal meta (for human play, based on my experience on what people enjoy about pvp games that aren’t purely emphasizing skill) is one that lacks an optimal strategy, because the usage of an optimal strategy implies its own downfall, and that this meta-countering operation is acyclic. (A cyclic meta is likely created by accident, and kills the pvp community if left as-is)

And notably, a random strategy selection being optimal is non-ideal for human consumption. And as you note this doesn’t naturally occur in human pvp, because there are heavy natural biases (information spread, natural leaders in the subject, limited skillsets, time for the community to learn between dev balance shifts, etc). But even if we could have it, I don’t think we’d want it.

I think what competitive pvp wants are somewhat obvious optimal solutions, with natural counter-play. But these near-optimal solutions are tied to the current popular strategy. That is, half the fun is figuring out what the community at large is up to, and tracking it.

Which, finally, implies that the kind of games that grow a significant pvp community are naturally selected because they offer no clear, and static, optimal strategy. If an ML program did find such a strategy (outside of requiring superhuman capabilities, like zerglings dodging siege tanks), it would either kill the community, or get patched out. You could consider the ML algorithm as competing with an adverserial meat learning algorithm, in both strategy and spirit

Post reply on HN