Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pac-Man recreated with a GAN trained on 50k game episodes

blogs.nvidia.com

Post reply on HN
Print view View original

51–60 of 154 posts

Re: Pac-Man recreated with a GAN trained on 50k game episodes

#51

Post by simias »

itchyjunk wrote: post #8

Thought experiment: Is my brain also generating reality without a physics engine and is filling in details that it thinks should be there? If true, what are the advantages/disadvantages of it?

The other answers to your comment are great but I also want to point out that you can feel your brain doing that if you play some videogames for a long time. You can feel that you develop an intuition for the way the game will behave even if the in-game physics are very different from our real word, showing the plasticity of our brain. If we had a fully hardcoded physics simulator in our head we couldn't "think with portals".

Re: Pac-Man recreated with a GAN trained on 50k game episodes

#52

Post by simias »

pbhjpbhj wrote: post #19

What're the total number of possible states in Pacman?

The number of possible pellet configurations on the grid is astronomical on its own, then you have pacman positions and ghost positions at least. I don't know the exact value but my estimate is "a very big number".

Re: Pac-Man recreated with a GAN trained on 50k game episodes

#53

Post by Terretta »

cooljacob204 wrote: post #20
QuixoticQuibit wrote: post #16

This research reminds me of one of their April Fool’s videos [1] with a gaming USB stick that you plug into your computer to offer various gaming enhancements. One of which is a “Ghostplay” feature where an AI program can mimic you after training on your gameplay. I remember that video because it always struck me as not being far off from reality. It’ll make the current set of online game cheats/hacks seem like child…

Servers are already running AI to try and counter cheating in quite a few games.

Aside, but, I'm said this is an acceptable use of the term AI, to mean "big statistics".

Re: Pac-Man recreated with a GAN trained on 50k game episodes

#54

Post by HeWhoLurksLate »

the_af wrote: post #49
coldpie wrote: post #10

Earlier quoted context omitted.

Yes, absolutely. You can catch a baseball because your brain predicts where it will go before your eyes see its actual movement. This may be a good place to start learning more: https://en.wikipedia.org/wiki/Direct_and_indirect_realism

Another fun example of how your brain predicts physics stuff is when you misjudge what something weighs. For example, if you subconsciously judge a glass is full of water but it's actually empty, as you try to lift it your arm will fly up awkwardly. This all happens without conscious thought. Don't tell me it never happened to you!

I knocked myself over once by lifting a solid double-corrugated cardboard box that I thought had a mini-lathe in it- I got ready to lift with both legs and promptly threw the box into my face and fell over.

That was one of the weirdest experiences I think I've had.

Re: Pac-Man recreated with a GAN trained on 50k game episodes

#55

Post by ki28iukgvdru »

ttul wrote: post #14
ekianjo wrote: post #13

> Trained on 50,000 episodes of the game Meanwhile an actual human will probably need only about 5 games to fully understand how the game works and reacts.

Fundamentally, this is because we have already been trained on years of living in and reacting to the real world, which, to some abstract degree, the game mimicks. The Nvidia model starts with nothing.

Indeed. Some work has actually been done to try to quantify how important various types of human prior knowledge are for learning video games: https://arxiv.org/pdf/1802.10217.pdf

Re: Pac-Man recreated with a GAN trained on 50k game episodes

#56

Post by tlarkworthy »

Terretta wrote: post #53
cooljacob204 wrote: post #20

Earlier quoted context omitted.

Servers are already running AI to try and counter cheating in quite a few games.

Aside, but, I'm said this is an acceptable use of the term AI, to mean "big statistics".

I find it amusing you seem sad because one hyped term was used, only to go on to prefix the word statistics with "big". Statistics was comfortable with large quantities since inception.

Re: Pac-Man recreated with a GAN trained on 50k game episodes

#57

Post by jerf »

lonelappde wrote: post #34
gallerdude wrote: post #30

I wonder what the copyright implications of this are. Obviously it's Pacman, but none of the source is the same.

Game rules aren't copyrightable. The visuals are a copy but obviously fair use, and it's obviously trademark use, and Nvidia probably has a marketing partnership with NAMCO for this

"“We were blown away when we saw the results, in disbelief that AI could recreate the iconic PAC-MAN experience without a game engine,” said Koichiro Tsutsumi from BANDAI NAMCO Research Inc., the research development company of the game’s publisher BANDAI NAMCO Entertainment Inc., which provided the PAC-MAN data to train GameGAN. “This research presents exciting possibilities to help game developers accelerate the creative process of developing new level layouts, characters and even games.”"

Emphasis mine.

Also, while I disagree that this would be real "fair use" (rather than Internet Pundit fair use, which is virtually indistinguishable from "I want this, so it must be legal") without permission, it wouldn't matter anyhow. If nVidia did this without permission and tried to distribute exactly what we saw, it would also be a trademark violation. Pacman is almost certainly in the top 1% of defended trademarks, if not .1% or .01%... not a great thing to try to use without permission.

Re: Pac-Man recreated with a GAN trained on 50k game episodes

#58

Post by laumars »

lonelappde wrote: post #34
gallerdude wrote: post #30

I wonder what the copyright implications of this are. Obviously it's Pacman, but none of the source is the same.

Game rules aren't copyrightable. The visuals are a copy but obviously fair use, and it's obviously trademark use, and Nvidia probably has a marketing partnership with NAMCO for this

I'm not convinced the visuals would be fair use. However it was certainly done in partnership with Namco. It's celebrating Pac-Man's 40th anniversary and says at the bottom of the article: PAC-MAN™ & ©BANDAI NAMCO Entertainment Inc.

Plus there is a massive give away that Namco approved this within the article:

We were blown away when we saw the results, in disbelief that AI could recreate the iconic PAC-MAN experience without a game engine,” said Koichiro Tsutsumi from BANDAI NAMCO Research Inc., the research development company of the game’s publisher BANDAI NAMCO Entertainment Inc., which provided the PAC-MAN data to train GameGAN. “This research presents exciting possibilities to help game developers accelerate the creative process of developing new level layouts, characters and even games.”

Re: Pac-Man recreated with a GAN trained on 50k game episodes

#59

Post by weinzierl »

QuixoticQuibit wrote: post #16

This research reminds me of one of their April Fool’s videos [1] with a gaming USB stick that you plug into your computer to offer various gaming enhancements. One of which is a “Ghostplay” feature where an AI program can mimic you after training on your gameplay. I remember that video because it always struck me as not being far off from reality. It’ll make the current set of online game cheats/hacks seem like child…

Something similar to what you describe is already being done. Game video is streamed to a Raspberry Pi Zero that sits between the real mouse and the Gaming PC and adjusts the mouse movements. Google for "Valorant RPi0" for videos to see this in action.

Here is an excerpt from a now deleted Reddit comment where the setup is described:

> "My pi is connected to my pc as a HID device with all the descriptors matching a legitimate mouse. My actual mouse is not connected to my pc and the movements are relayed through to the pi over wifi (UDP). My pc records my screen and streams info to my pi over wifi which my pi happily adjusts as seen in the video."

Re: Pac-Man recreated with a GAN trained on 50k game episodes

#60

Post by CShorten »

bjourne wrote: post #48
CShorten wrote: post #4

I made a video explaining the details of this paper if you are interested! https://youtu.be/H8F6J7mYyz0

Very interesting video! How does the GameGAN image discriminator work? I don't know so much about GANs but it seem to me that classifying PacMan images as real or fake would be a very hard problem because of the low resolution. Suppose PacMan is in a corridor and there is a dot to the left of him and three to the right: ".P..." That is plausible because PacMan hasn't eaten any dots yet. But if the second dot to the r…

Thanks for watching bjourne! There are three different discriminators, the first does realism for the game, the second does action-conditioned discrimination (making sure the generator takes in the action for its next frame prediction), and the third is a temporal realism with a 3D CNN.

Interesting scenario, it would be a cool experiment to walk through the latent space and try to find that kind of frame in the GAN latent space. I'm sure you could find it, but the question might be how far off the generated data's manifold is it.

Post reply on HN