This guy has been making music controlled LED items, boxes and wrist bands.
https://www.kickstarter.com/projects/markusloeffler/lumiband...
Audio Reactive LED Strips Are Diabolically Hard
31–40 of 76 posts
Re: Audio Reactive LED Strips Are Diabolically Hard
#32I like this writeup but I feel like the title doesn't really tell you what it's about ... to me it's about creativity within constraints. The author finds, as many do, that naive or first-approximation approaches fail within certain constraints and that more complex methods are necessary to achieve simplicity. He finds, as I have, that perceptual and spectral domains are a better space to work in for things that are…
Yeah, "diabolical" overstates it. It isn't a wicked problem https://en.wikipedia.org/wiki/Wicked_problem Kinda funny but I am a fan of green LED light to supplement natural light on hot summer days. I can feel the radiant heat from LED lights on my bare skin and since the human eye is most sensitive to green light I feel the most comfortable with my LED strip set to (0,255,0)
I'd actually argue it has some wicked problem characteristics. The input space is enormous (all possible audio), perception is subjective and nonlinear, and there's no objective function to optimize against, only "does this feel right?". Every solution you try reframes what "good" means. It's not as hard as social planning but is way harder than it sounds, no pun intended.
Re: Audio Reactive LED Strips Are Diabolically Hard
#33Woow, this was my first hardware project right around the time it released! I remember stapling a bunch of LED strips around our common room and creating a case for the pi + power supply by drilling a bunch of ventilation + cable holes in a wooden box. And of course, by the time I got it to work perfectly I never looked at it again. As is tradition.
That's awesome to hear! Sometimes the journey is the destination, its a great project to get started with electronics.
Re: Audio Reactive LED Strips Are Diabolically Hard
#34The real killer is that humans don't hear frequencies, they hear instruments, which are a stack of frequencies that roughly sometimes correlate with a frequency range.
I wonder if transformer tech is close to achieving real-time audio decoding, where you can split a track into it's component instruments, and light show off of that. Think those fancy Christmas time front yard light shows as opposed to random colors kind of blinking with what maybe is a beat.
Re: Audio Reactive LED Strips Are Diabolically Hard
#35Interesting. I'm currently in the process of building something with a audio reactive LED strip but didn't come across this project yet. The WLED [1] ESP32 firmware seems to be able to do something similar or potentially more though. [1] https://kno.wled.ge/ Edit: Oh wait, that project needs a PC or Raspberry PI for audio processing. WLED does everything on the ESP32.
WLED is decent but tbh the lag is very noticeable. Did you compare to this python thing?
No, haven't tried it.
For my use case I want something fully portable and battery powered anyways. So the audio stuff should happen on the ESP32. (Or on my phone, that might work too)
Re: Audio Reactive LED Strips Are Diabolically Hard
#36It's not that hard. I did a real-time version of the Beatroot algorithm decades ago that worked pretty well for being such a simple algorithm.
Re: Audio Reactive LED Strips Are Diabolically Hard
#37I like this writeup but I feel like the title doesn't really tell you what it's about ... to me it's about creativity within constraints. The author finds, as many do, that naive or first-approximation approaches fail within certain constraints and that more complex methods are necessary to achieve simplicity. He finds, as I have, that perceptual and spectral domains are a better space to work in for things that are…
I didn't go into much detail about it but there's a whole rabbit hole of color theory and color models. For example, the spectrum effect assigns different colors to different frequency bins, but also adjusts the assignment over time to avoid a static looking effect. It does this by rotating a "color angle" kind of like the HSL model. I really like your LED installation in Rosetta Hall, it looks beautiful!
Thanks! Great article - would like to read one about the color rabbit hole pls ;)
Re: Audio Reactive LED Strips Are Diabolically Hard
#38Earlier quoted context omitted.
Yeah, "diabolical" overstates it. It isn't a wicked problem https://en.wikipedia.org/wiki/Wicked_problem Kinda funny but I am a fan of green LED light to supplement natural light on hot summer days. I can feel the radiant heat from LED lights on my bare skin and since the human eye is most sensitive to green light I feel the most comfortable with my LED strip set to (0,255,0)
I'd actually argue it has some wicked problem characteristics. The input space is enormous (all possible audio), perception is subjective and nonlinear, and there's no objective function to optimize against, only "does this feel right?". Every solution you try reframes what "good" means. It's not as hard as social planning but is way harder than it sounds, no pun intended.
Ever seen https://www.youtube.com/watch?v=oNyXYPhnUIs ? There are a lot of things people might think feels right.
(Note both the scanner in front of KITT and the visual FX on his dashboard when he speaks, which changes from season to season.)
Re: Audio Reactive LED Strips Are Diabolically Hard
#39This is awesome! I did a similar project in college for one of my classes and ran into the same exact walls as you.
- The more filters I added the worse it got. A simple EMA with smoothing gave the best results. Although, your pipeline looks way better than what I came up with!
- I ended up using the Teensy 4.0 which let me do real time FFT and post processing in less than 10ms (I want to say it was ~1ms but I can't recall; it's been a while). If anyone goes down this path I'd heavily recommend checking out the teensy. It removes the need for a raspi or computer. Plus, Paul is an absolute genius and his work is beyond amazing [1].
- I started out with non-addressable LEDs also. I attempted to switch to WS2812's as well, but couldn't find a decent algorithm to make it look good. Yours came out really well! Kudos.
- Putting the leds inside of an LED strip diffuser channel made the biggest difference. I spent so long trying to smooth it out getting it to look good when a simple diffuser was all I needed (I love the paper diffuser you made).
RE: What's Still Missing: I came to a similar conclusion as well. Manually programmed animation sequences are unparalleled. I worked as a stagehand in college and saw what went into their shows. It was insane. I think the only way to have that same WOW factor is via pre-processing. I worked on this before AI was feasible, but if I were to take another stab at it I would attempt to do it with something like TinyML. I don't think real time is possible with this approach. Although, maybe you could buffer the audio with a slight delay? I know what I'll be doing this weekend... lol.
Again, great work. To those who also go down this rabbit hole: good luck.
Re: Audio Reactive LED Strips Are Diabolically Hard
#40The real killer is that humans don't hear frequencies, they hear instruments, which are a stack of frequencies that roughly sometimes correlate with a frequency range. I wonder if transformer tech is close to achieving real-time audio decoding, where you can split a track into it's component instruments, and light show off of that. Think those fancy Christmas time front yard light shows as opposed to random colors ki…
real time audio stem separation is already possible, some specific models can even get around 20ms latency (HS-TasNet) https://github.com/lucidrains/HS-TasNet
There was a nice paper with an overview last year too https://arxiv.org/html/2511.13146v1 that introduced RT-STT which is still being tweaked and built upon in the MSS scene
The high quality ones like MDXNet and Demucs usually have at least several seconds of latency though, but for something like displaying visuals high quality is not really needed and the real time approaches should be fine.