Very cool! As a further variation on this idea, I'm imagining training a reinforcement learning agent on atari games / super mario, but with an additional music-based reward/input, to try to get a "musical" looking playthrough... (Not sure how good it would look / whether it would be worth it though...)
Synchronizing Pong to music with constrained optimization
31–40 of 40 posts
Re: Synchronizing Pong to music with constrained optimization
#32https://en.wikipedia.org/wiki/Atari_Video_Music
If you've seen the movie Over the Edge, Claude and Johnny have one at their house.
Re: Synchronizing Pong to music with constrained optimization
#33Re: Synchronizing Pong to music with constrained optimization
#34While technically okay, there are multiple cases where a paddle and the ball move at the almost identical speed and it looks like the paddle is pushing the ball all the time. (By the way, p[i] = 0 should be disallowed for this reason.) This is inevitable when a large d[i] is immediately followed by a very small d[i+1], but it might be able to avoid them whenever feasible.
Re: Synchronizing Pong to music with constrained optimization
#35Now try synchronising the music to the game.
You could use our Bungee library for the audio processing.
Re: Synchronizing Pong to music with constrained optimization
#36Very cool! As a further variation on this idea, I'm imagining training a reinforcement learning agent on atari games / super mario, but with an additional music-based reward/input, to try to get a "musical" looking playthrough... (Not sure how good it would look / whether it would be worth it though...)
The original game’s sound was tied to the frame rate so this vaguely happened by default. Later ports to PAL broke this because it ran at a slower frame rate.
Re: Synchronizing Pong to music with constrained optimization
#37Re: Synchronizing Pong to music with constrained optimization
#38This reminds a bit of a project I coded the audio for an art exhibition around 20 years ago, which was for multiple simultaneous players. As the game got more intense it became apparent that the ball/wall sounds were playing music, and that the three players were all actually playing one musical piece. It was based around 3 arcade cabinets pointing together, so the players couldn't see what was on each others screens…
Re: Synchronizing Pong to music with constrained optimization
#39Prior art: Eisenfunk - Pong ( https://www.youtube.com/watch?v=cNAdtkSjSps )
Re: Synchronizing Pong to music with constrained optimization
#40This reminds a bit of a project I coded the audio for an art exhibition around 20 years ago, which was for multiple simultaneous players. As the game got more intense it became apparent that the ball/wall sounds were playing music, and that the three players were all actually playing one musical piece. It was based around 3 arcade cabinets pointing together, so the players couldn't see what was on each others screens…
That’s neat. So it was a static composition that the gameplay was tweaked into fitting?
The audio engine (the bit I worked on) was in effect a stem based mixer, but with a state transition diagram per stem, with multiple loops available. Depending on the route through the transitions (triggered by events from the game play, for example, how many balls the player is keeping in play) it was possible to reach more complex or simpler performances of the same musical piece, so the players decisions and ability would affect how the performance of the music, not the music itself if that makes sense?