A hidden gem in sound symmetry
soundshader.github.io
A hidden gem in sound symmetry
1–10 of 54 posts
Re: A hidden gem in sound symmetry
#2Hi HN, author here. A few comments on how I came up with this idea. I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images. The latter part is hard, as perception of "musical harmony" is vaguely defined and subjective. Nevertheless, after quite a few attempts, I came across a particularly simple FFT-based technique that produces impressive and unexpected results. Below is the summary of my finding.
Music is a temporal ornament. There are many types of ornaments, e.g. the 17 types of wallpaper tesselations, but few of them look like music. However there is one particular type of ornament that resembles music a lot - I mean those “mandala” images. I don’t know how those are produced, but I noticed a connection between those images and music:
- The 1st obvious observation is that a mandala is drawn in polar coordinates and is 2PI periodic. Sound is periodic too, so I thought the two facts are related.
- The 2nd observation is that patterns on those images evolve over the radial axis. Ans so is music is a sequence of evolving sound patterns.
- The 3rd observation is that a 2PI periodic function trivially corresponds to a set of frequencies. We usually use FFT to extract the frequencies and another FFT to restore the 2PI periodic function. Thus, a single radial slice of a mandala could encode a set of frequencies. If this is correct, a mandala is effectively an old school vinyl disk.
Putting these observations together, we naturally arrive with the idea of using ACF. More details in the linked github project.
Re: A hidden gem in sound symmetry
#3Hi HN, author here. A few comments on how I came up with this idea. I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images. The latter part is hard, as per…
I'm guessing that triple correlation could also be rendered into pretty pictures, did you try that?
Re: A hidden gem in sound symmetry
#4beautiful. What's the demo soundtrack?
Re: A hidden gem in sound symmetry
#5Hi HN, author here. A few comments on how I came up with this idea. I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images. The latter part is hard, as per…
I'm guessing that triple correlation could also be rendered into pretty pictures, did you try that?
Heh, you're reading my mind. Try the URL below, but don't increase fps/n params, as that will eat all your GPU cycles very quickly:
soundshader.github.io/?s=acf3&n=512&fps=1&acf.decay=0
It effectively computes the bispectrum as B(p, q) = F(p)F(q)F^(p+q) and runs the inverse 2D FFT to restore the triple autocorrelation. The results are interesting, but not impressive and very GPU intensive (NxNxlog(N) per frame is slow). In any case, I strongly believe that bispectrum is hiding something interesting and I just haven't figured how to see it.
Re: A hidden gem in sound symmetry
#6beautiful. What's the demo soundtrack?
It's a file from freesound that captures the male "A" vowel, but slowed down like 20x.
Re: A hidden gem in sound symmetry
#7Autocorrelation has been used in psychoacoustics at least since Licklider's work in the 1950s. But I'm not sure if I've seen this style of visualization before. It looks a bit like the output of a strobe tuner.
Re: A hidden gem in sound symmetry
#8Would love to see this applied to financial price signals.
Re: A hidden gem in sound symmetry
#9Hi HN, author here. A few comments on how I came up with this idea. I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images. The latter part is hard, as per…
> I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images.
It is very exciting to come across others who are also interested in this topic. I am also very interested in the shape of sound but I have spent less time on empirical observations and more on imagining an abstract logic of numbers which can be visualized and heard. Real sound visualizations are also interesting to me but I decided to focus on abstract ideals because I thought it would be appropriate for a video game.
Hope you don't mind me sending some emails.
Re: A hidden gem in sound symmetry
#10Hi HN, author here. A few comments on how I came up with this idea. I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images. The latter part is hard, as per…
> I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images. It is very exciting to come across others who are also interested in this topic. I am also very i…
Your emails are welcome! As for a physical sound visualization, one exists already: see "Numerical simulation of Faraday waves" by L. Tuckerman. As usual, the dry technical paper glosses over the visual aspect of Faraday waves. Basically, if you create such a standing wave in a cup of tea (e.g. by shaking the table), you'd see its shape via reflections, similar to how we can see the surface of ocean waves via reflections of sun. However, if you could suspend the water surface, put a LED ring above (one of those used for professional photos) and take an extremely high quality photo to see all the reflections and refractions created by that LED ring, yo'd see a picture of remarkable complexity. It looks a lot like a 3D hologram. I've been trying to simulate this effect on GPU.