Earlier this year, graphic designers, last month it was software engineers, and now musicians are also feeling the effects. Who else will AI make looking for a new job?
Riffusion – Stable Diffusion fine-tuned to generate music
31–40 of 481 posts
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#32This is a genius idea. Using an already-existing and well-performing image model, and just encoding input/output as a spectrogram... It's elegant, it's obvious in retrospective, it's just pure genius. I can't wait to hear some serious AI music-making a few years from now.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#33Earlier this year, graphic designers, last month it was software engineers, and now musicians are also feeling the effects. Who else will AI make looking for a new job?
GPT-3, what policy should we apply to increase tax revenue by 5% given these constraints?
GPT-3, please tell me some populist thing to say to win the next election, or how should I deflect these corruption charges.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#34This is a genius idea. Using an already-existing and well-performing image model, and just encoding input/output as a spectrogram... It's elegant, it's obvious in retrospective, it's just pure genius. I can't wait to hear some serious AI music-making a few years from now.
The amazing thing is that the current diffusion models are so good that the spectograms are actually reasonable enough despite the small room for error.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#35This really is unreasonably effective. Spectrograms are a lot less forgiving of minor errors than a painting. Move a brush stroke up or down a few pixels, you probably won't notice. Move a spectral element up or down a bit and you have a completely different sound. I don't understand how this can possibly be precise enough to generate anything close to a cohesive output. Absolutely blows my mind.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#36Really cool. Can't get this to work on the homepage though. Might be a traffic thing? Edit: Works now. A bit laggy but it works. Brilliant!
{"data":{"success":true,"worklet_output":{"error":"Model version 5qekv1q is not healthy"},"latency_ms":530}}
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#37It absolutely sucks at cymbals, though. Everything sounds like realaudio :) composition's lacking, too. It's loop-y.
Set this up to make AI dubtechno or trip-hop. It likes bass and indistinctness and hypnotic repetitiveness. Might also be good at weird atonal stuff, because it doesn't inherently have any notion of what a key or mode is?
As a human musician and producer I'm super interested in the kinds of clarity and sonority we used to get out of classic albums (which the industry has kinda drifted away from for decades) so the way for this to take over for ME would involve a hell of a lot more resolution of the FFT imagery, especially in the highs, plus some way to also do another AI-ification of what different parts of the song exist (like a further layer but it controls abrupt switches of prompt)
It could probably do bad modern production fairly well even now :) exaggeration, but not much, when stuff is really overproduced it starts to get way more indistinct, and this can do indistinct. It's realaudio grade, it needs to be more like 128kbps mp3 grade.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#38Re: Riffusion – Stable Diffusion fine-tuned to generate music
#39Earlier this year, graphic designers, last month it was software engineers, and now musicians are also feeling the effects. Who else will AI make looking for a new job?