I'm very interested in this, but I can't seem to get the demo page working in latest Firefox: > AudioContext.createMediaStreamSource: Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported. Edge worked though. Didn't try Chrome. Edit: This is beautiful! I have checked out a variety of songs, and I feel the visualization is rather dominated by whatever frequency is loudest. (E.g…
A hidden gem in sound symmetry
21–30 of 54 posts
Re: A hidden gem in sound symmetry
#22Seems to work really well! Playing Gasoline by Audioslave, it really misses the high intensity peaks. It interprets the trumpet in Miles Davis - It Never Entered my Mind as dark ripples and is beautiful in it's own way. It's a long way from the last time I used a visualizer on Winamp, nice!
Edit. The generated images in fact contain the small ripples, but our eyes don't notice the 0.1% modulation of color. If they did, we'd see bright orange-blue waves with a fine pattern of wavelets colored with a slightly different shade of orange and blue. People who can see 100 shades of orange would see this pattern.
Re: A hidden gem in sound symmetry
#23Images reminded me of cymatic patterns. I wonder if there is any relation there
Re: A hidden gem in sound symmetry
#24Earlier quoted context omitted.
It's a file from freesound that captures the male "A" vowel, but slowed down like 20x.
Why the slowdown? (given that slowdown typically introduces artifacting, making the result as much a pattern of those artifacts as it is of the original signal) (also note that your site does not currently work in Firefox, which would be nice to fix)
The demo works on Firefox 78, Ubuntu. However you'd have to set correct sample rate with ?sr=44.1 to match the mp3's sample rate: Firefox won't do resampling.
Re: A hidden gem in sound symmetry
#25Hi HN, author here. A few comments on how I came up with this idea. I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images. The latter part is hard, as per…
Right now, the visual experience is like watching movement through a high-speed tunnel. I was expecting the "mandala" you mentioned in the sense that the end result is the accumulated visualization of all waves.
The sound representation would not disappear out of the borders. The first sound recorded would be stored as a narrow outside ring right next to the circle limit. The next sound would be stored as another narrow ring right before the first one. And it would continuously being accumulated sound after sound. The final result would be like a tree cut. It would have a final image representation the whole song, not just sequential snapshots of the sounds included in the song as it is now.
Anyway, congrats for the project! It is awesome and inspiring!
Re: A hidden gem in sound symmetry
#26Hi HN, author here. A few comments on how I came up with this idea. I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images. The latter part is hard, as per…
I checked the live demo with some music. I was expecting something different and maybe you agree that it would be a better visualization. Right now, the visual experience is like watching movement through a high-speed tunnel. I was expecting the "mandala" you mentioned in the sense that the end result is the accumulated visualization of all waves. The sound representation would not disappear out of the borders. The f…
Re: A hidden gem in sound symmetry
#27Hi HN, author here. A few comments on how I came up with this idea. I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images. The latter part is hard, as per…
Chladni Plates? A few links from past research:
https://www.comsol.com/blogs/how-do-chladni-plates-make-it-p...
Video example - https://www.youtube.com/watch?v=dPTnGEEoFf4
https://www.youtube.com/watch?v=CR_XL192wXw
interactive version - https://www.dynamicmath.xyz/calculus/chladni-patterns/
Re: A hidden gem in sound symmetry
#28Re: A hidden gem in sound symmetry
#29I really enjoyed your work: the write-up was clear and the demo page worked well.
Re: A hidden gem in sound symmetry
#30Hi HN, author here. A few comments on how I came up with this idea. I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images. The latter part is hard, as per…
> those "Mandala" images Chladni Plates? A few links from past research: https://www.comsol.com/blogs/how-do-chladni-plates-make-it-p... Video example - https://www.youtube.com/watch?v=dPTnGEEoFf4 https://www.youtube.com/watch?v=CR_XL192wXw interactive version - https://www.dynamicmath.xyz/calculus/chladni-patterns/ https://www.youtube.com/watch?v=tFAcYruShow
(1) Making a stable and fast solver is very difficult. A simple solver for the canonical wave equation is fast, but unstable, so the solution has to be periodically adjusted to avoid NaNs. A stable solver, even for the simplest equation, would be 20x slower. For complex cases, we'd have to involve the Floquet theory, but that would bring an already slow solver to a halt.
(2) A wave diff equation can barely visualize a select frequency, not even a simple mix of frequencies or let alone music. The thing is such wave equations and their boundary shapes have a few select "resonance frequencies" that produce semi-stable patterns. Even a tiny step from a stable frequency, e.g. 6.1 Hz vs 6 Hz, and the solution turns into a mix of unstable patterns morphing one into another, which is cool, but not visually appealing. Mixing multiple frequencies together often produces an unstable mess and even if a pattern forms, you're never sure if it's the pattern for that frequency or just a transient shape, and if it's transient, you can't know if it's due to numeric errors or due to the nature of the equation.
(3) Limited resolution. The rule of thumb is that on a 1000x1000 px screen, the densest Chladni pattern would make 500 full wave repetitions, one pixel per positive and negative sides of the wave. This means we can render only the 500 different frequencies, with 500 Hz slowly turning into a mess due to rounding errors (2 pixels per wave period isn't really enough). Increasing the internal solver buffer to say 4096px brings fps down to 3-4.
However, despite all this, Chladni patterns are hiding something very remarkable, that seems to be glossed over in technical papers. If you imagine that a Chaldni pattern is a water or glass surface, with reflective and refractive properties, and look at the reflection of a simple symmetric object, e.g. a ring, you'd see something resembling a 3d hologram: all these inter-reflections will produce a "virtual image" or remarkable complexity. This picture can be taken by a hi-res camera, but visualizing it with GLSL is again very difficult: the raytracer needs to be outrageously precise.