21–30 of 46 posts
can this be done on real hardware with a modified controller?
https://www.youtube.com/watch?v=OJXxmD_Qk2o This is a video of it running on a console, though I'm uncertain how the inputs were provided
I thought the audio was just overlayed on top, but it was streamed in via the controller. It sounds AMAZING, incredibly even on the console!
Earlier quoted context omitted.
Hey, I'm the TASer who put this run together. This was 5.8 million inputs. I share the full assembly code in the tasvideos writeup: https://tasvideos.org/8991S#HereSTheAsmCode To summarize what I put in the writeup, the 7-bit PCM audio was streamed in at approximately 25 Khz, (reading from the controller and writing to address $4011 every 71 CPU cycles.) while occasionally dipping to 9 Khz while streaming in the grap…
I never really got the part about the audio conversion; are you just rounding it from 16 to 7 bits, or are you doing dithering + noise shaping (as you definitely should at such low bit depths)? And similarly, how are you downsampling from 44 kHz to a time-varying sample rate; are you properly filtering, or are you getting tons of aliasing?
My method of downsampling was complicated. Since the creation of the TAS was being automated, and since I also needed to stream in graphics data occasionally, I ran into the issue of needing to know exactly what byte to read from the .wav file at any given moment. I used a custom NES emulator to emulate the generated inputs, and I had it count CPU cycles so I can convert that into seconds, then parse the .wav file with that info.
To be completely honest, this project was my first time directly reading the contents of a .wav file like this, and I had no prior experience writing code for audio conversion or playback. If I were to do this project again, I'd look into noise dithering + noise shaping, as well as filtering methods. I know at the very end of the TAS, there's certainly some weird audio artifacts that I couldn't figure out how to fix at the time.
Super cool. Is it mentioned anywhere how big the payload is? How many button presses? Are the audio samples "streamed" or does it all fit in NES RAM?
Hey, I'm the TASer who put this run together. This was 5.8 million inputs. I share the full assembly code in the tasvideos writeup: https://tasvideos.org/8991S#HereSTheAsmCode To summarize what I put in the writeup, the 7-bit PCM audio was streamed in at approximately 25 Khz, (reading from the controller and writing to address $4011 every 71 CPU cycles.) while occasionally dipping to 9 Khz while streaming in the grap…
Earlier quoted context omitted.
Hey, I'm the TASer who put this run together. This was 5.8 million inputs. I share the full assembly code in the tasvideos writeup: https://tasvideos.org/8991S#HereSTheAsmCode To summarize what I put in the writeup, the 7-bit PCM audio was streamed in at approximately 25 Khz, (reading from the controller and writing to address $4011 every 71 CPU cycles.) while occasionally dipping to 9 Khz while streaming in the grap…
I'd love a laymane overview of what you've done here
1. Get the game into a specific state by performing specific actions, moving to specific positions, performing specific inputs, etc. so that a portion of the game state in RAM happens to be an executable program.
2. Jump to that executable code such as by corrupting the return address in the stack with a buffer overflow
3. (optional) The program from 1 may be a simple "bootstrap" program which lets the player directly write a new, larger program using controller inputs then jumps to the new program.
4. The program reads the video and audio from the stream of controller inputs, decodes them, and displays them. The encoding is usually an ad-hoc scheme designed to take advantage of the available hardware. The stream of replayed inputs is computed directly from the media files.
can this be done on real hardware with a modified controller?
https://www.youtube.com/watch?v=OJXxmD_Qk2o They use a modified ROM to set the memory state, but that could be done with SMB3 manually, apparently.
Earlier quoted context omitted.
https://www.youtube.com/watch?v=OJXxmD_Qk2o This is a video of it running on a console, though I'm uncertain how the inputs were provided
Usually TASes use a modified controller with wires hooked up directly to the controller's circuit board.
Where is the data coming from? Why does it involve playing part of SMB to start with? Besides the sprites used, what does this have to do with SMB at all? How is this different from just a custom ROM that would play this music video? Could you even fit this much data into a ROM?
I feel like there's a bunch of context you need to understand what this is about, and I clearly don't have any of it. And the "objectives" list at the top isn't helping ("1. Do a little tomfoolery 2. Execute arbitrary code..."). What are the actual parameters of this challenge, to understand what is actually being achieved?
Can anybody here help explain?
Edit: thank you so much for all the replies! Now I get why this is wild.
Earlier quoted context omitted.
I never really got the part about the audio conversion; are you just rounding it from 16 to 7 bits, or are you doing dithering + noise shaping (as you definitely should at such low bit depths)? And similarly, how are you downsampling from 44 kHz to a time-varying sample rate; are you properly filtering, or are you getting tons of aliasing?
The conversion from 16 bits to 7 bits was using rounding. My method of downsampling was complicated. Since the creation of the TAS was being automated, and since I also needed to stream in graphics data occasionally, I ran into the issue of needing to know exactly what byte to read from the .wav file at any given moment. I used a custom NES emulator to emulate the generated inputs, and I had it count CPU cycles so I…
As a very quick fix, you can dither by just adding a random value from [-0.5, +0.5] before rounding (to -64..+63 or whatever your range is). It will give you a dither, and probably sound slightly better; a bit more noise for much less distortion. Noise shaping is left as an exercise for the reader :-) (It is probably nontrivial to get perfect with variable sample rate anyway.)
> I used a custom NES emulator to emulate the generated inputs, and I had it count CPU cycles so I can convert that into seconds, then parse the .wav file with that info.
It sounds like you are just picking one sample without any filtering/averaging/anything (nearest neighbor); this will cause aliasing, which is another part of the reason for the “roughness” you may hear in the sound. You can do a very cheap trick here as well: Take some audio software you trust (say, Audacity) and convert the .wav file to 25208 Hz. This means that you'll get good filtering for most of your audio, and less bad filtering for the 13.85kHz parts.
I've read the post but I still have no idea what this is. Where is the data coming from? Why does it involve playing part of SMB to start with? Besides the sprites used, what does this have to do with SMB at all? How is this different from just a custom ROM that would play this music video? Could you even fit this much data into a ROM? I feel like there's a bunch of context you need to understand what this is about,…
This video is done by hand and explains a bit more about how and why it works: https://www.youtube.com/watch?v=hB6eY73sLV0