Live data from Hacker News

A hidden gem in sound symmetry

soundshader.github.io

21–30 of 54 posts

Re: A hidden gem in sound symmetry

#21

I'm very interested in this, but I can't seem to get the demo page working in latest Firefox: > AudioContext.createMediaStreamSource: Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported. Edge worked though. Didn't try Chrome. Edit: This is beautiful! I have checked out a variety of songs, and I feel the visualization is rather dominated by whatever frequency is loudest. (E.g…

It works on Firefox, but as the error says, you have to set the correct sample rate that matches the mp3 file. You can do that with ?sr=44.1 or ?sr=48. The default value is 44.1 and on Chrome you can use any sample rate.

Re: A hidden gem in sound symmetry

#22

Seems to work really well! Playing Gasoline by Audioslave, it really misses the high intensity peaks. It interprets the trumpet in Miles Davis - It Never Entered my Mind as dark ripples and is beautiful in it's own way. It's a long way from the last time I used a visualizer on Winamp, nice!

One problem with the current color scheme is what I'd call the dynamic range of volumes. If the input is a mix of very loud and very quiet waves, as often the case in club music, its ACF will look like a stormy ocean: huge slow waves with a lot of ripples on them. ACF doesn't lose this information, but visualizing it is difficult.

Edit. The generated images in fact contain the small ripples, but our eyes don't notice the 0.1% modulation of color. If they did, we'd see bright orange-blue waves with a fine pattern of wavelets colored with a slightly different shade of orange and blue. People who can see 100 shades of orange would see this pattern.

Re: A hidden gem in sound symmetry

#23

Images reminded me of cymatic patterns. I wonder if there is any relation there

Both share the same idea, but math is certainly different. Those cymatics patterns are solutions of the u_tt = u_xx + u_yy equation with t modulated by a sine wave and the images themselves are reflections of light in the u(x, y) surface.

Re: A hidden gem in sound symmetry

#24
post #6

Earlier quoted context omitted.

It's a file from freesound that captures the male "A" vowel, but slowed down like 20x.

Why the slowdown? (given that slowdown typically introduces artifacting, making the result as much a pattern of those artifacts as it is of the original signal) (also note that your site does not currently work in Firefox, which would be nice to fix)

It's just a random sound sample I found on freesound.

The demo works on Firefox 78, Ubuntu. However you'd have to set correct sample rate with ?sr=44.1 to match the mp3's sample rate: Firefox won't do resampling.

Re: A hidden gem in sound symmetry

#25
post #2

Hi HN, author here. A few comments on how I came up with this idea. I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images. The latter part is hard, as per…

I checked the live demo with some music. I was expecting something different and maybe you agree that it would be a better visualization.

Right now, the visual experience is like watching movement through a high-speed tunnel. I was expecting the "mandala" you mentioned in the sense that the end result is the accumulated visualization of all waves.

The sound representation would not disappear out of the borders. The first sound recorded would be stored as a narrow outside ring right next to the circle limit. The next sound would be stored as another narrow ring right before the first one. And it would continuously being accumulated sound after sound. The final result would be like a tree cut. It would have a final image representation the whole song, not just sequential snapshots of the sounds included in the song as it is now.

Anyway, congrats for the project! It is awesome and inspiring!

Re: A hidden gem in sound symmetry

#26
post #25
post #2

Hi HN, author here. A few comments on how I came up with this idea. I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images. The latter part is hard, as per…

I checked the live demo with some music. I was expecting something different and maybe you agree that it would be a better visualization. Right now, the visual experience is like watching movement through a high-speed tunnel. I was expecting the "mandala" you mentioned in the sense that the end result is the accumulated visualization of all waves. The sound representation would not disappear out of the borders. The f…

If you're on desktop, try moving your mouse vertically. "Up" seems to zoom in, and "Down" zooms out. Fully zoomed-out, I think it's closer to what you expected to see.

Re: A hidden gem in sound symmetry

#27
post #2

Hi HN, author here. A few comments on how I came up with this idea. I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images. The latter part is hard, as per…

> those "Mandala" images

Chladni Plates? A few links from past research:

https://www.comsol.com/blogs/how-do-chladni-plates-make-it-p...

Video example - https://www.youtube.com/watch?v=dPTnGEEoFf4

https://www.youtube.com/watch?v=CR_XL192wXw

interactive version - https://www.dynamicmath.xyz/calculus/chladni-patterns/

https://www.youtube.com/watch?v=tFAcYruShow

Re: A hidden gem in sound symmetry

#29
Somehow I have memories of the 1990s and audio visualisations coming back... Fantastic. I'd love to be able to plug this into mpv for playing audio files. I wonder how hard this would be (or how many resources it would use...).

I really enjoyed your work: the write-up was clear and the demo page worked well.

Re: A hidden gem in sound symmetry

#30
post #2

Hi HN, author here. A few comments on how I came up with this idea. I've been trying to find a "proper" connection between audible sound and visible shape, a connection that would not only preserve all the information, but would also properly visualize the "symmetry" in sound, so that messy sound would turn into messy images and harmonic sound would turn into visually appealing images. The latter part is hard, as per…

> those "Mandala" images Chladni Plates? A few links from past research: https://www.comsol.com/blogs/how-do-chladni-plates-make-it-p... Video example - https://www.youtube.com/watch?v=dPTnGEEoFf4 https://www.youtube.com/watch?v=CR_XL192wXw interactive version - https://www.dynamicmath.xyz/calculus/chladni-patterns/ https://www.youtube.com/watch?v=tFAcYruShow

Yeah, that's where my journey started. I've explored those quite a bit, even wrote a GLSL solver (wave-glsl.github.io/web) to visualize height maps of the u_tt = u_xx + u_yy equation (there is a shallow water equation solver there as well). Problems were that:

(1) Making a stable and fast solver is very difficult. A simple solver for the canonical wave equation is fast, but unstable, so the solution has to be periodically adjusted to avoid NaNs. A stable solver, even for the simplest equation, would be 20x slower. For complex cases, we'd have to involve the Floquet theory, but that would bring an already slow solver to a halt.

(2) A wave diff equation can barely visualize a select frequency, not even a simple mix of frequencies or let alone music. The thing is such wave equations and their boundary shapes have a few select "resonance frequencies" that produce semi-stable patterns. Even a tiny step from a stable frequency, e.g. 6.1 Hz vs 6 Hz, and the solution turns into a mix of unstable patterns morphing one into another, which is cool, but not visually appealing. Mixing multiple frequencies together often produces an unstable mess and even if a pattern forms, you're never sure if it's the pattern for that frequency or just a transient shape, and if it's transient, you can't know if it's due to numeric errors or due to the nature of the equation.

(3) Limited resolution. The rule of thumb is that on a 1000x1000 px screen, the densest Chladni pattern would make 500 full wave repetitions, one pixel per positive and negative sides of the wave. This means we can render only the 500 different frequencies, with 500 Hz slowly turning into a mess due to rounding errors (2 pixels per wave period isn't really enough). Increasing the internal solver buffer to say 4096px brings fps down to 3-4.

However, despite all this, Chladni patterns are hiding something very remarkable, that seems to be glossed over in technical papers. If you imagine that a Chaldni pattern is a water or glass surface, with reflective and refractive properties, and look at the reflection of a simple symmetric object, e.g. a ring, you'd see something resembling a 3d hologram: all these inter-reflections will produce a "virtual image" or remarkable complexity. This picture can be taken by a hi-res camera, but visualizing it with GLSL is again very difficult: the raytracer needs to be outrageously precise.

Post reply on HN