I propose that while you are GPU limited, you make these changes:
* Don't do the alpha fade to the next prompt - just jump straight to alpha=1.0.
* Pause the playback if the server hasn't responded in time, rather than looping.
Riffusion – Stable Diffusion fine-tuned to generate music
111–120 of 481 posts
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#112Other author here! This got a posted a little earlier than we intended so we didn't have our GPUs scaled up yet. Please hang on and try throughout the day! Meanwhile, please read our about page http://riffusion.com/about It’s all open source and the code lives at https://github.com/hmartiro/riffusion-app --> if you have a GPU you can run it yourself This has been our hobby project for the past few months. Seeing the…
All the AI music I’ve heard so far has a really unpleasant resonant quality to it. Why is that? Can it be removed?
You're probably talking about the artifacts of converting a low resolution spectrogram to audio.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#113This is a genius idea. Using an already-existing and well-performing image model, and just encoding input/output as a spectrogram... It's elegant, it's obvious in retrospective, it's just pure genius. I can't wait to hear some serious AI music-making a few years from now.
Makes me wonder if we will see a generalization of this idea. Just like in a CPU 90%+ of want you want to do can be modeled with very few instructions (mov, add, jmp..) we could see a set of very refined models (Stable difussion, GPT, etc) and all of their abstractions on top (ChatGPT, Rifussion, etc).
Indeed, I think this would be a cost-effective way to go forward.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#114Other author here! This got a posted a little earlier than we intended so we didn't have our GPUs scaled up yet. Please hang on and try throughout the day! Meanwhile, please read our about page http://riffusion.com/about It’s all open source and the code lives at https://github.com/hmartiro/riffusion-app --> if you have a GPU you can run it yourself This has been our hobby project for the past few months. Seeing the…
All the AI music I’ve heard so far has a really unpleasant resonant quality to it. Why is that? Can it be removed?
It sounds kind of like the visual artifacts that are generated by resampling in two dimensions. Since the whole model is based on compressing image content, whatever it's doing DSP-wise is more-or-less "baked in", and a probable fix would lie in doing it in a less hacky way.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#115Other author here! This got a posted a little earlier than we intended so we didn't have our GPUs scaled up yet. Please hang on and try throughout the day! Meanwhile, please read our about page http://riffusion.com/about It’s all open source and the code lives at https://github.com/hmartiro/riffusion-app --> if you have a GPU you can run it yourself This has been our hobby project for the past few months. Seeing the…
All the AI music I’ve heard so far has a really unpleasant resonant quality to it. Why is that? Can it be removed?
The link is down now, so I don't know about this one. But most generated music is generated in the note domain, rather than the audio domain. Any unpleasant resonance would introduced in the audio synthesis step. And audio synthesis from note data is a very solved problem for any kind of timbre you can conceive of, and some you can't.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#116I got an actual `HTTP 402: PAYMENT_REQUIRED` response (never seen one of those in the wild, according to Mozilla it is experimental). Someone's credit card stopped scaling?
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#117Earlier quoted context omitted.
Presumably for similar reasons that the vast majority of AI generated art and text is off-puttingly hideous or bland. For every stunning example that gets passed around the internet, thousands of others sucked. Generating art that is aesthetically pleasing to humans seems like the Mt. Everest of AI challenges to me.
I think your comment is off-topic to the post you are replyng to. That wasn't asking about the general aesthetic quality - more about a specific audio artifact. > For every stunning example that gets passed around the internet, thousands of others sucked. From personal experience this is simply untrue. I don't want to debate it because you seem to have strong feelings about the topic.
Even if you remove the artifact, the exact same comment applies. It generates a somewhat less interesting version of elevator music. This is not to crap on what they did. As I said, they underlying problem is extremely difficult and nobody has managed to solve it.
I don't feel strongly about this topic at all.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#118This is a genius idea. Using an already-existing and well-performing image model, and just encoding input/output as a spectrogram... It's elegant, it's obvious in retrospective, it's just pure genius. I can't wait to hear some serious AI music-making a few years from now.
> I can't wait to hear some serious AI music-making a few years from now. I think this will be particularly useful for musical compositions in movies and film, where the producer can "instruct" the AI about what to play, when, and how to transition so that the music matches the scene progression.
I was thinking gaming where music can and should dynamically shift based on different environmental and player conditions.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#119Earlier quoted context omitted.
All the AI music I’ve heard so far has a really unpleasant resonant quality to it. Why is that? Can it be removed?
Presumably for similar reasons that the vast majority of AI generated art and text is off-puttingly hideous or bland. For every stunning example that gets passed around the internet, thousands of others sucked. Generating art that is aesthetically pleasing to humans seems like the Mt. Everest of AI challenges to me.
The vast majority of human generated art is hideous or bland. Artists throw away bad ideas or sketches that didn’t work all the time. Plus you should see most of the stuff that gets pasted up on the walls at an average middle
School.
Re: Riffusion – Stable Diffusion fine-tuned to generate music
#120If it can do music, can we train better models for different kinds of music? Or different models for different instruments makes more sense? For different instruments we can get better resolution by making the spectrogram represent different frequency ranges. This is terribly exciting, what a time to be alive.