Riffusion – Stable Diffusion fine-tuned to generate music
1–10 of 481 posts
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#2This is so good that I wondered if it's fake. Really impressive results from generated spectrographs! Also really interesting that it's not exactly trained on the audio files themselves - wonder if the usual copyright-based objections wild even apply here.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#3Really cool. Can't get this to work on the homepage though.
Might be a traffic thing?
Edit: Works now. A bit laggy but it works. Brilliant!
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#4Really cool. Can't get this to work on the homepage though. Might be a traffic thing? Edit: Works now. A bit laggy but it works. Brilliant!
[deleted]
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#5This is so good that I wondered if it's fake. Really impressive results from generated spectrographs! Also really interesting that it's not exactly trained on the audio files themselves - wonder if the usual copyright-based objections wild even apply here.
[deleted]
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#6Very impressive. I am quite confident that next years number one Christmas hit will start like "church bells to electronic beats".
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#7Producing images of spectrograms is a genius idea. Great implementation!
A couple of ideas that come to mind:
- I wonder if you could separate the audio tracks of each instrument, generate separately, and then combine them. This could give more control over the generation. Alignment might be tough, though.
- If you could at least separate vocals and instrumentals, you could train a separate model for vocals (LLM for text, then text to speech, maybe). The current implementation doesn't seem to handle vocals as well as TTS models.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#8Really cool. Can't get this to work on the homepage though. Might be a traffic thing? Edit: Works now. A bit laggy but it works. Brilliant!
I also don't hear anything, even when my prompt was selected...
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#9great stuff, while it comes with the usual smeary iFFT artifacts that AI-generated sound tends to have the results are surprisingly good.
i especially love the nonsense vocals it generates in the last example, which remind me of what singing along to foreign songs felt like in my childhood.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#10impressive stuff. reminds me of when ppl started using image classifier networks on spectrograms in order to classify audio. i would not have thought to apply a similar concept for generative models, but it seems obvious in hindsight.