I will try it but at least for the name it deserves praise.
Riffusion – Stable Diffusion fine-tuned to generate music
81–90 of 481 posts
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#82Other author here! This got a posted a little earlier than we intended so we didn't have our GPUs scaled up yet. Please hang on and try throughout the day! Meanwhile, please read our about page http://riffusion.com/about It’s all open source and the code lives at https://github.com/hmartiro/riffusion-app --> if you have a GPU you can run it yourself This has been our hobby project for the past few months. Seeing the…
When you say fine tuned do you mean fine tuned on an existing stable diffusion checkpoint? If so which? It would be very interesting to see what the stable diffusion community that is using automatic1111 version would do with this if it were made into an extension.
Yes from https://huggingface.co/runwayml/stable-diffusion-v1-5. Our checkpoint works with automatic1111, and if you'd like to make an extension to decode to audio, it should be pretty straightforward: https://github.com/hmartiro/riffusion-inference/blob/main/ri...
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#83Other author here! This got a posted a little earlier than we intended so we didn't have our GPUs scaled up yet. Please hang on and try throughout the day! Meanwhile, please read our about page http://riffusion.com/about It’s all open source and the code lives at https://github.com/hmartiro/riffusion-app --> if you have a GPU you can run it yourself This has been our hobby project for the past few months. Seeing the…
[deleted]
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#84Was just watching an interview of Billy Corgan (smashing pumpkins) on Rick Beato’s YouTube[1] last night where billy was lamenting the inevitable future where the “psychopaths” in the music biz will use ai and auto tune to churn out three chord non-music mumble rap for the youth of tomorrow, or something to that effect. It was funny because it’s the sad truth. It’s already here but new tech will allow them to cut costs even more, and increase their margins. No need for musicians. Really cool on one hand, in the same way fentanyl is cool — or the cotton gin, but a bit depressing on the other, if you care about musicians. I and a few others will always pay to go the symphony, so good players will find a way get paid, but this is what kids will listen to, because of the profit margin alone.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#85Can anyone confirm/deny my theory that AI audio generation has been lagging behind progress in image generation because it’s way easier to get a billion labeled images than a billion labeled audio clips?
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#86This is really cool but can someone tell me why we are automating art? Who asked for this? The future seems depressing when I look at all this AI generated art.
Everyone asked for this, including artists. If you make a living off of making art, having the best tools to help you do that is a constant, and the tools are finally starting to get properly good. Will "the job" change because of the tools? Of course. Will the nature of what it means for something to be art change? Also of course. Art isn't some static, untouchable thing. It changes as humanity does.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#87Earlier quoted context omitted.
When you say fine tuned do you mean fine tuned on an existing stable diffusion checkpoint? If so which? It would be very interesting to see what the stable diffusion community that is using automatic1111 version would do with this if it were made into an extension.
Yes from https://huggingface.co/runwayml/stable-diffusion-v1-5 . Our checkpoint works with automatic1111, and if you'd like to make an extension to decode to audio, it should be pretty straightforward: https://github.com/hmartiro/riffusion-inference/blob/main/ri...
Can you run this on any hardware already capable of running SD 1.5? I am downloading the model right now, might play with this later.
Guessing at the speed with which AI is developing these days someone is going to have the extension up in two hours at most.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#88Earlier quoted context omitted.
Yes from https://huggingface.co/runwayml/stable-diffusion-v1-5 . Our checkpoint works with automatic1111, and if you'd like to make an extension to decode to audio, it should be pretty straightforward: https://github.com/hmartiro/riffusion-inference/blob/main/ri...
Can you run this on any hardware already capable of running SD 1.5? I am downloading the model right now, might play with this later. Guessing at the speed with which AI is developing these days someone is going to have the extension up in two hours at most.
I bet the AUTOMATIC1111 web UI music plugin drops within 48 hours.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#89Earlier quoted context omitted.
Yes from https://huggingface.co/runwayml/stable-diffusion-v1-5 . Our checkpoint works with automatic1111, and if you'd like to make an extension to decode to audio, it should be pretty straightforward: https://github.com/hmartiro/riffusion-inference/blob/main/ri...
Can you run this on any hardware already capable of running SD 1.5? I am downloading the model right now, might play with this later. Guessing at the speed with which AI is developing these days someone is going to have the extension up in two hours at most.
Yes! Although to have real time playback with our defaults you need to be able to generate 40 steps at 512x512 in under 5 seconds.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#90Some of this is really cool! The 20 step interpolations are very special, because they're concepts that are distinct and novel. It absolutely sucks at cymbals, though. Everything sounds like realaudio :) composition's lacking, too. It's loop-y. Set this up to make AI dubtechno or trip-hop. It likes bass and indistinctness and hypnotic repetitiveness. Might also be good at weird atonal stuff, because it doesn't inhere…
> composition's lacking, too. It's loop-y.
Well no wonder, it has absolutely no concept of composition beyond a single 5s loop, if I understand correctly.
> It absolutely sucks at cymbals, though. Everything sounds like realaudio :)
> It could probably do bad modern production fairly well even now :) exaggeration, but not much, when stuff is really overproduced it starts to get way more indistinct, and this can do indistinct. It's realaudio grade, it needs to be more like 128kbps mp3 grade.
I haven't sat down yet to calculate it, but is the output of SD at 512*512px at 24bit enough to generate audio CD quality in theory?