Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Riffusion – Stable Diffusion fine-tuned to generate music

riffusion.com

Post reply on HN
Print view View original

401–410 of 481 posts

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#401

Post by Agentlien »

xtracto wrote: post #66

This looks great and the idea is amazing. I tried with the prompt: "speed metal" and "speed metal with guitar riffs" and got some smooth rock-balad type music. I guess there was no heavy metal in the learning samples haha. Great work!

It does seem to lack a lot in heavy music in general. My first attempt was to get it to generate something akin to AC/DC and it got fairly close but it still seemed a bit too clean and pop-like. Then I tried to get something closer to nu metal or deathcore and it just kept generating some smooth upbeat jazz. Which is not that bad in its own right, but not at all what I asked for.

Edit: After a bit of playing around I at least got some credible results with "electric guitar solo, glam rock". It also understands grunge.

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#402

Post by Agentlien »

razemio wrote: post #388
bawolff wrote: post #305

Earlier quoted context omitted.

Honestly none of them should. I think the moral panic around these things is way overstated. They are cool but hardly about to replace anyone's job.

Have you tried AI asset generators? They are working extremely good. Just yesterday a friend of mine has shown me the progress they made in their game. It is incredible. Designers are 100% loosing their job over this.

I'm a professional game developer and excited AI enthusiast.

While I've seen a lot of cool stuff which helps generation for hobby projects or smaller indie games it's nowhere near the quality and consistency needed to come close to the work of a skilled human artist at a larger studio.

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#403

Post by andybak »

itronitron wrote: post #267
TaupeRanger wrote: post #248

Earlier quoted context omitted.

It's...not effective though. Am I listening to the wrong thing here? Everything I hear from the web app is jumbled nonsense.

I think we're at the point, with these AI generative model thingies, where the practitioners are mesmerized by the mechatronic aspect like a clock maker who wants to recreate the world with gears, so they make a mechanized puppet or diorama and revel in their ingenuity.

And that's a bad thing?

How do you think human endeavours progress other than by small steps?

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#404

Post by JofArnold »

munificent wrote: post #254
bheadmaster wrote: post #19

This is a genius idea. Using an already-existing and well-performing image model, and just encoding input/output as a spectrogram... It's elegant, it's obvious in retrospective, it's just pure genius. I can't wait to hear some serious AI music-making a few years from now.

As someone who loves making music and loves listening to music made by other humans with intention, it just makes me sad. Sure, AI can do lots of things well. But would you rather live in a world where humans get to do things they love (and are able to afford a comfortable life while doing so) or a world where machines do the things humans love and humans are relegated to the remaining tasks that machines happened to…

Musicians already make much (most?) of their money via gigs and I don't think going to watch an AI play at a gig will be all too common. I think we'll be fine. Might have to adapt though.

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#405

Post by jonathanstrange »

WheelsAtLarge wrote: post #317
jtode wrote: post #247

Earlier quoted context omitted.

As one of the meatsacks whose job you're about to kill... eh, I got nothin, it's damn impressive. It's gonna hit electronic music like a nuclear bomb, I'd wager.

These are tools. Don't think of them as replacements, they aren't. But as tools that will help us be creative. As smart as these apps seem, they will still need a human to decide where and how to use them. They won't replace us but we need to adapt to a new reality.

These will be full replacements in no time, give or take 10 years.

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#406

Post by fragmede »

jonathanstrange wrote: post #405
WheelsAtLarge wrote: post #317

Earlier quoted context omitted.

These are tools. Don't think of them as replacements, they aren't. But as tools that will help us be creative. As smart as these apps seem, they will still need a human to decide where and how to use them. They won't replace us but we need to adapt to a new reality.

These will be full replacements in no time, give or take 10 years.

10 years‽ StableDiffusion was released August 22nd of this year!

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#407

Post by navane »

owlbynight wrote: post #281

If copyright laws don't catch up, the sampling industry is cooked. Made this: https://soundcloud.com/obnmusic/ai-sampling-riffusion-waves-...

assuming the first sample was generated by OP method, how did you clean that sample up so nicely?

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#408

Post by O__________O »

>> Prompt - When providing prompts, get creative! Try your favorite artists, instruments like saxophone or violin, modifiers like arabic or jamaican, genres like jazz or rock, sounds like church bells or rain, or any combination. Many words that are not present in the training data still work because the text encoder can associate words with similar semantics. The closer a prompt is in spirit to the seed image And BPM, the better the results. For example, a prompt for a genre that is much faster BPM than the seed image will result in poor, generic audio.

(1) Is there a corpse the keywords were collected from?

(2) Is it possible to model the proximity of the image to keywords and sets of keywords?

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#409

Post by teruakohatu »

ozten wrote: post #77
haykmartiros wrote: post #59

Other author here! This got a posted a little earlier than we intended so we didn't have our GPUs scaled up yet. Please hang on and try throughout the day! Meanwhile, please read our about page http://riffusion.com/about It’s all open source and the code lives at https://github.com/hmartiro/riffusion-app --> if you have a GPU you can run it yourself This has been our hobby project for the past few months. Seeing the…

Amazing work! Did you use CLIP or something like that to train genre + mel-spectrogram? What datasets did you use?

I was very surprised this was not mentioned.

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#410

Post by neuronic »

rob74 wrote: post #394
spoiler wrote: post #258

Earlier quoted context omitted.

As a listener, I think you're probably still safe. Can you use this to help you though? Maybe. It's impressive what it produces, but I think it probably lacks substance in the same way the visual AI art stuff does. For the most part, it passes what I call the at-a-glanceness test. It's little better than apophenia (the same thing that makes you see shapes in clouds, faces in rocks, or think you've recognised a famili…

I fully agree with what you wrote. This AI-generated music, while a great achievement, still sounds soulless. It's one thing to look at AI-generated pictures for a few seconds, but listening to this music with its gibberish "lyrics" for minutes really creeps me out - it's the "uncanny valley" all over again, I guess. Regarding "can you use this to help you through?" - yeah, you could probably use it as a source of in…

Yea, it's uncanney valley, sure. For now.

With Stable Diffusion and similar generative systems we have seen a leap in generative art/media, partially with significant improvements within a few months. What makes you think this was the last or only leap in the next 5 to 10 years? As if progress would just stop here? Huh?!

Do you think we hit a ceiling were progress is only tangential? A line which is impossible to cross? Otherwise I dont get this mindset in the face of these modern generative AI systems popping up left and right.

Post reply on HN