GPT-3 has 175 billion parameters (says Wikipedia). What is the size of the neural network used in this riffusion project?
Stable Diffusion 'only' has ~1B parameters IIRC.
Riffusion – Stable Diffusion fine-tuned to generate music
71–80 of 481 posts
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#72BOOM! Yes!
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#73I bet a cool riff on this would be to simply sample an ambient microphone in the workplace and use that the generate and slowly introduce matching background music that fits the current tenor of the environment. Done slowly and subtly enough I'd bet the listener may not even be entirely aware its happening. If we could measure certain kinds of productivity it might even be useful as a way to "extend" certain highly p…
>in the workplace
Or at a house party, club or restaurant... as more people arrive or leave and the energy level rises or declines..or human rhythms speed up or slow down...so does the music...
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#74This is really cool but can someone tell me why we are automating art? Who asked for this? The future seems depressing when I look at all this AI generated art.
You can't automate a live performance or an oil painting with AI in this way. This isn't going to replace musicians and artists. If anything, I think a preponderance of AI art would make people appreciate the real stuff more. As to why, music is fun to create, and this is just a tool.
> You can't automate a live performance or an oil painting with AI in this way.
You'd have to combine it with these guys https://www.youtube.com/watch?v=WqE9zIp0Muk
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#75This is a genius idea. Using an already-existing and well-performing image model, and just encoding input/output as a spectrogram... It's elegant, it's obvious in retrospective, it's just pure genius. I can't wait to hear some serious AI music-making a few years from now.
I'm super excited about the Audio AI space, as it seems permanently a few years behind image stuff - so I think we're going to see a lot more of this.
If you're interested, the idea of applying Image processing techniques to Spectrograms of audio is explored in brief in the first lesson of one of the most recommended AI courses on HN: Practical Deep Learning for Coders https://youtu.be/8SF_h3xF3cE?t=1632
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#76I bet a cool riff on this would be to simply sample an ambient microphone in the workplace and use that the generate and slowly introduce matching background music that fits the current tenor of the environment. Done slowly and subtly enough I'd bet the listener may not even be entirely aware its happening. If we could measure certain kinds of productivity it might even be useful as a way to "extend" certain highly p…
>in the workplace Or at a house party, club or restaurant... as more people arrive or leave and the energy level rises or declines..or human rhythms speed up or slow down...so does the music...
DJs are getting automated away too!
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#77Other author here! This got a posted a little earlier than we intended so we didn't have our GPUs scaled up yet. Please hang on and try throughout the day! Meanwhile, please read our about page http://riffusion.com/about It’s all open source and the code lives at https://github.com/hmartiro/riffusion-app --> if you have a GPU you can run it yourself This has been our hobby project for the past few months. Seeing the…
Amazing work! Did you use CLIP or something like that to train genre + mel-spectrogram? What datasets did you use?
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#78Things similar to the “interpolation” part (not the generative part) are already used extensively especially for game and movie sound design. Kyma [1] is the absolute leader (it requires expensive hardware though). IMO later iterations on this approach may lead to similar or better results.
FYI, other apps that use more classic but still complex Spectral/Granular algos :
https://www.thecargocult.nz/products/envy
https://transformizer.com/products/
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#79I’m curious about the limitations of using spectrograms and transient-heavy sounds like drums.
It seems like you’d need very high resolution spectrograms to get a consistently snappy drum sound.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#80[deleted]