What did use as training data?
Riffusion – Stable Diffusion fine-tuned to generate music
141–150 of 481 posts
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#142Other author here! This got a posted a little earlier than we intended so we didn't have our GPUs scaled up yet. Please hang on and try throughout the day! Meanwhile, please read our about page http://riffusion.com/about It’s all open source and the code lives at https://github.com/hmartiro/riffusion-app --> if you have a GPU you can run it yourself This has been our hobby project for the past few months. Seeing the…
The audio sounds a bit lossy, would it be possible to create high quality spectograms from music, downsample them, and use that as training data for a spectogram upscaler?
It might be the last step this AI needs to bring some extra clarity to the output.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#143why not use an image of the waveform as input?
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#144This looks great and the idea is amazing. I tried with the prompt: "speed metal" and "speed metal with guitar riffs" and got some smooth rock-balad type music. I guess there was no heavy metal in the learning samples haha. Great work!
Gregorian death metal folk also seems to have lacked seed tunes but the thing is just in its infancy so soon we'll be banging our tonsured heads to the folky beats of ...
...OK, need to create a band name generator to work in tandem with this thing. Let's see what one of its brethren in ML makes of it...
- "Echoes of the Past": This name plays on the idea of Gregorian chanting, which is often associated with the distant past, and combines it with the intense and aggressive sound of death metal.
- "The Order of the Black Chant": This name incorporates elements of both the religious connotations of Gregorian chanting and the dark, heavy sound of death metal, creating a sense of mystery and danger.
- "Foretold in Blood": This name evokes both the ancient, mystical nature of Gregorian chanting and the violent themes of death metal, creating a sense of ancient prophecy coming to pass.
- "Crypt of the Silent Choir": This name brings together the eerie, otherworldly sound of Gregorian chanting with the underground, underground feel of death metal, creating a sense of hidden secrets and forbidden knowledge.
"The Order of the Black Chant" it shall be.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#145"https://en.wikipedia.org/wiki/Spectrogram - can we already do sound via image? probably soon if not already"
Me in the Stable Diffusion discord, 10/24/2022
The ppl saying this was a genius idea should go check out my other ideas
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#146deleted
I think you'll find plenty of people who find that DAWs and music theory help them better find self-expression and celebrate life through their music. Any tool or framework that opens up new modes of achieving that self-expression should be celebrated, not shunned because it isn't as "pure" as more time and labor intensive methods. Would you rather someone be forced to dedicate a significant amount of time to studyin…
[deleted]
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#147Earlier quoted context omitted.
All the AI music I’ve heard so far has a really unpleasant resonant quality to it. Why is that? Can it be removed?
I've done some work on AI audio synthesis and the artifacts you're hearing in these clips are coming from the algorithm that is used to go from the synthesized spectrogram to the audio (the Griffin-Lim algorithm). Audio spectrograms have two components: the magnitude and the phase. Most of the information and structure is in the magnitude spectrogram so neural nets generally only synthesize that. If you were to look…
Griffin-Lim is slow and is almost certainly not being used.
A neural vocoder such as Hifi-Gan [1] can convert spectra to audio - not just for voices. Spectral inversion works well for any audio domain signal. It's faster and produces much higher quality results.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#148Earlier quoted context omitted.
You're probably talking about the artifacts of converting a low resolution spectrogram to audio.
Can the spectrogram image be AI upscaled before transforming back to the time domain?
Yes it exists: https://ccrma.stanford.edu/~juhan/super_spec.html
But the issue is not that the spectrogram is low quality.
The issue is that the spectrogram only contains the amplitude information. You also need phase information for generating audio from the spectogram
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#149Earlier quoted context omitted.
All the AI music I’ve heard so far has a really unpleasant resonant quality to it. Why is that? Can it be removed?
I've done some work on AI audio synthesis and the artifacts you're hearing in these clips are coming from the algorithm that is used to go from the synthesized spectrogram to the audio (the Griffin-Lim algorithm). Audio spectrograms have two components: the magnitude and the phase. Most of the information and structure is in the magnitude spectrogram so neural nets generally only synthesize that. If you were to look…
I'm curious why, instead of using magnitude and phase, you wouldn't use real and imaginary parts?
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#150" https://en.wikipedia.org/wiki/Spectrogram - can we already do sound via image? probably soon if not already" Me in the Stable Diffusion discord, 10/24/2022 The ppl saying this was a genius idea should go check out my other ideas
If only we had a diffusional model that could take your ideas and turn them into reality!