GAN Theft Auto [video]
youtube.com
GAN Theft Auto [video]
1–10 of 99 posts
Re: GAN Theft Auto [video]
#2Impressive. Makes you wonder if at some point in the future there isn't a game engine any more but tons of training material and you play in a generated dream.
Re: GAN Theft Auto [video]
#3Impressive. Makes you wonder if at some point in the future there isn't a game engine any more but tons of training material and you play in a generated dream.
Certainly impressive. And sure, maybe in a distant future. Though I think this is like one of those things where creating a working prototype that is 75% complete is the "easy" part. The other 25% (which you need for an actual working product) will take forever. Like self driving cards, nuclear fusion, etc.
Re: GAN Theft Auto [video]
#4I fail to see novelty here. What's the size difference between the model and and all of the 64x32 image training data? If the difference is not significant, you're basically almost just scrubbing a video, right?
Re: GAN Theft Auto [video]
#5Someone did a similar project with the exact same name for an ML art project at CMU a few years ago.
https://m.youtube.com/watch?v=eP5hHKne_gE&feature=youtu.be
Full list of projects: https://sites.google.com/site/artml2018/showcase/final-proje...
Re: GAN Theft Auto [video]
#6Someone did a similar project with the exact same name for an ML art project at CMU a few years ago. https://m.youtube.com/watch?v=eP5hHKne_gE&feature=youtu.be Full list of projects: https://sites.google.com/site/artml2018/showcase/final-proje...
Jeez, scared me. Same name yep, totally different project. That project is pix2pix. That is not a GAN-based game engine that you play within.
Re: GAN Theft Auto [video]
#7I fail to see novelty here. What's the size difference between the model and and all of the 64x32 image training data? If the difference is not significant, you're basically almost just scrubbing a video, right?
The GAN model is the game environment. You're playing a neural network. The novelty is no game engine, no rules, just learned how to represent the game and you can play it.
Re: GAN Theft Auto [video]
#8Someone did a similar project with the exact same name for an ML art project at CMU a few years ago. https://m.youtube.com/watch?v=eP5hHKne_gE&feature=youtu.be Full list of projects: https://sites.google.com/site/artml2018/showcase/final-proje...
Jeez, scared me. Same name yep, totally different project. That project is pix2pix. That is not a GAN-based game engine that you play within.
Oh, yeah, definitely a different thing but kinda neat that the name has occurred twice. Sorry for the scare!
Re: GAN Theft Auto [video]
#9I fail to see novelty here. What's the size difference between the model and and all of the 64x32 image training data? If the difference is not significant, you're basically almost just scrubbing a video, right?
The GAN model is the game environment. You're playing a neural network. The novelty is no game engine, no rules, just learned how to represent the game and you can play it.
What he meant is you overfitted the network with video footage. There is no game, just seemingly clever stitching and playback of learned footage
similar concept applied to animations and implemented in a state machine https://www.youtube.com/watch?v=KSTn3ePDt50
and optimized with nn https://www.youtube.com/watch?v=16CHDQK4W5k
Re: GAN Theft Auto [video]
#10Great work! Hacker News still seems to have a deeply skeptical culture with regard to machine learning - not sure why. There's always someone saying it's "not novel" and it's "just doing x".
Overfitting is a known issues in machine learning, people. If you still think all neural networks are doing is memorizing the dataset completely in the year 2021 - you might want to revisit the topic. It is one of the first concerns anyone training a deep model will have and to assume this model is overfit _without_ providing specific examples is arguing in bad faith.
Sentdex has shown his GAN is able to generalize various game logic like collision/friction with vehicles and also learns aspects of rendering such as a proper reflection of the sun on the back of the car.
He also showed weak points where the model is incapable of handling some situations and even did the impossible task of "splitting a car in two" to try and solve a head-on collision. Even though this is a failure case; it should at least provide you with some intuition that the GAN isn't just spitting out frames memorized from the dataset because that never happens in the dataset.
You will need to apply a little more rigor before outright dismissing these weights as merely overfit.
@sentdex Have you considered a guided diffusion approach now that that's all the rage? It's all rather new still but I believe it could be applied to these concepts as well.