Pretty boring if it's just FFT based. I'd love to see music visualizations make use of more sophisticated musical feature detection, like polyphonic note recognition or tempo/rhythm analysis.
Real-Time Music Visualization on the iPhone GPU
11–16 of 16 posts
Re: Real-Time Music Visualization on the iPhone GPU
#12Pretty boring if it's just FFT based. I'd love to see music visualizations make use of more sophisticated musical feature detection, like polyphonic note recognition or tempo/rhythm analysis.
Hey, I'm Adrian. I did the 'More colors' shader.
Given the nature of the project, a hackathon, sticking to the last computed FFTs and a notion of time was all we need to play with nice shader effects.
I'm currently playing with tensorflow on my spare time. I'd like to use a neural net to find interesting features of a track and inputting this to a shader. That's what I would do if I had another go at it.
Re: Real-Time Music Visualization on the iPhone GPU
#13Pretty boring if it's just FFT based. I'd love to see music visualizations make use of more sophisticated musical feature detection, like polyphonic note recognition or tempo/rhythm analysis.
Hey, I'm Adrian. I did the 'More colors' shader. Given the nature of the project, a hackathon, sticking to the last computed FFTs and a notion of time was all we need to play with nice shader effects. I'm currently playing with tensorflow on my spare time. I'd like to use a neural net to find interesting features of a track and inputting this to a shader. That's what I would do if I had another go at it.
Cool work, but note that a 1024 point FFT isn't really sufficient to visualize music. At 44.1kHz, the lowest frequency you can resolve is 43Hz. Anything below that will be folded into your DC component. Your bass response is mostly going to be overall volume, rather than beat.
Re: Real-Time Music Visualization on the iPhone GPU
#14Earlier quoted context omitted.
Hey, I'm Adrian. I did the 'More colors' shader. Given the nature of the project, a hackathon, sticking to the last computed FFTs and a notion of time was all we need to play with nice shader effects. I'm currently playing with tensorflow on my spare time. I'd like to use a neural net to find interesting features of a track and inputting this to a shader. That's what I would do if I had another go at it.
Cool work, but note that a 1024 point FFT isn't really sufficient to visualize music. At 44.1kHz, the lowest frequency you can resolve is 43Hz. Anything below that will be folded into your DC component. Your bass response is mostly going to be overall volume, rather than beat.
Most music is mixed not to have any useful spectral information below that frequency anyway. And if you want beat then a better thing you can do is look at the half-wave rectified difference between the sums of two FFT frames (with optional lowpass). This results in an onset graph that is the basis for a lot of real-time beat detection algorithms.