181–190 of 481 posts
This is a genius idea. Using an already-existing and well-performing image model, and just encoding input/output as a spectrogram... It's elegant, it's obvious in retrospective, it's just pure genius. I can't wait to hear some serious AI music-making a few years from now.
You already hear a ton of them. Lofi music on these massively popular channels are basically auto-generated "music" + auto generated artwork.
This is really cool but can someone tell me why we are automating art? Who asked for this? The future seems depressing when I look at all this AI generated art.
Because tons of people want to make art, and a lot of art currently requires years of training to make anything close to "good". Making art more accessible to create is a boon to everyone who's dreamed of being able to make their own paintings and music, but doesn't have the skills required.
Personally I like the results. I'm totally untrained and couldn't hear any of the issues many comments are pointing out.
I guess that all of lounge/elevator music and probably most ad jingles will be automated soon, if automation cost less than human authors.
This really is unreasonably effective. Spectrograms are a lot less forgiving of minor errors than a painting. Move a brush stroke up or down a few pixels, you probably won't notice. Move a spectral element up or down a bit and you have a completely different sound. I don't understand how this can possibly be precise enough to generate anything close to a cohesive output. Absolutely blows my mind.
Wasn't this Fraunhofer's big insight that led to the development of MP3? Human perception actually is pretty forgiving of perturbations in the Fourier domain.
Earlier quoted context omitted.
All the AI music I’ve heard so far has a really unpleasant resonant quality to it. Why is that? Can it be removed?
Presumably for similar reasons that the vast majority of AI generated art and text is off-puttingly hideous or bland. For every stunning example that gets passed around the internet, thousands of others sucked. Generating art that is aesthetically pleasing to humans seems like the Mt. Everest of AI challenges to me.
A musician friend of mine told me that this is (I freely translate) a perversion, building in frequency and returning time. Don't shoot the messenger. Personally I like the results. I'm totally untrained and couldn't hear any of the issues many comments are pointing out. I guess that all of lounge/elevator music and probably most ad jingles will be automated soon, if automation cost less than human authors.
Earlier quoted context omitted.
You're probably talking about the artifacts of converting a low resolution spectrogram to audio.
Can the spectrogram image be AI upscaled before transforming back to the time domain?
Earlier quoted context omitted.
Wasn't this Fraunhofer's big insight that led to the development of MP3? Human perception actually is pretty forgiving of perturbations in the Fourier domain.
You probably mean Karlheinz Brandenburg, the developer of MP3, who worked on psychoacoustics. Not completely off though, as he did the research at a Fraunhofer research institute, which takes its name from Joseph von Fraunhofer, the inventor of the spectroscope.
I think there has to be a better way to make long songs... For example, you could take half the previous spectrogram, shift it to the left, and then use the inpainting algorithm to make the next bit... Do that repeatedly, while smoothly adjusting the prompt, and I think you'd get pretty good results. And you could improve on this even more by having a non-linear time scale in the spectrograms. Have 75% of the image b…
But perhaps plain stable diffusion wouldn't work - you might need different neural networks trained on each "zoom level" because the structure would vary: music generally isn't like fractals and doesn't have exact self-similarity.
I'm quite impressed that there was enough training data within SD to know what a spectrograph looks like for the different sounds.