Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Riffusion – Stable Diffusion fine-tuned to generate music

riffusion.com

Post reply on HN
Print view View original

321–330 of 481 posts

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#321

Post by emporas »

tomrod wrote: post #136

This is huge. This show me that Stable Diffusion can create anything with the following conditions: 1. Can be represented as as static item on two dimensions (their weaving together notwithstanding, it is still piece-by-piece statically built) 2. Acceptable with a certain amount of lossiness on the encoding/decoding 3. Can be presented through a medium that at some point in creation is digitally encoded somewhere. Th…

Any image generator can do well in any two dimensional data, including SD, Dalle, Midjourney.

One feature of SD not discussed much in my opinion, is the deterministic key it provides to the user. This is what enables the smooth transition in every second of music it generates, and the next second in time. Moving the cursor of latent space between in a minimal way, creating the next piece of information and change it ever so slightly, it definitely sounds good to the human ears.

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#322

Post by jacquesm »

api wrote: post #260
spoiler wrote: post #258

Earlier quoted context omitted.

As a listener, I think you're probably still safe. Can you use this to help you though? Maybe. It's impressive what it produces, but I think it probably lacks substance in the same way the visual AI art stuff does. For the most part, it passes what I call the at-a-glanceness test. It's little better than apophenia (the same thing that makes you see shapes in clouds, faces in rocks, or think you've recognised a famili…

In general all this stuff is chopping the bottom off the market. AI art, code, writing, music, etc. can all generate passable "filler" content, which will decimate all human employment generating same. I don't think this stuff is a threat to genuinely innovative, thoughtful, meaningful work, but that's the top of the market. That being said the bottom of the market is how a lot of artists make their living, so this i…

> Basic income or revolution. That's going to be our choice.

I fear you are right. But neither of those is going to be an easy transition, if only because the effects of all this innovation is felt disproportionally by people in countries where such a revolution will not do anything to give solace.

Basic income assumes that the funds to do this are available and revolution assumes that the powers that be are the parties that are in the way of a more equitable division of the spoils. Neither of those are necessarily true for all locations affected.

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#323

Post by DoreenMichele »

api wrote: post #260
spoiler wrote: post #258

Earlier quoted context omitted.

As a listener, I think you're probably still safe. Can you use this to help you though? Maybe. It's impressive what it produces, but I think it probably lacks substance in the same way the visual AI art stuff does. For the most part, it passes what I call the at-a-glanceness test. It's little better than apophenia (the same thing that makes you see shapes in clouds, faces in rocks, or think you've recognised a famili…

In general all this stuff is chopping the bottom off the market. AI art, code, writing, music, etc. can all generate passable "filler" content, which will decimate all human employment generating same. I don't think this stuff is a threat to genuinely innovative, thoughtful, meaningful work, but that's the top of the market. That being said the bottom of the market is how a lot of artists make their living, so this i…

Basic income or revolution. That's going to be our choice.

Evolution.

We have such vast wealth and our historic methods for trying to make sure most people are taken care of are failing us. Those methods were rooted in the nuclear family with a head of household earning most of the money and jobs designed with an assumption that he had a full-time homemaker wife buying the groceries, cooking the meals etc so he could focus on his job.

We need jobs to evolve. In the US at least, we need to move away from tying all benefits (such as medical benefits and retirement) to a primary earner. We need to make it possible to live a comfortable life without a vehicle. We need to make it possible for small households to find small homes that make sense for them, both financially and in terms of lifestyle.

There is a lot we can do to make this not a disaster and make it possible for some people to survive on very little while while pursuing their bliss so that we stop this trend of pitting The Haves against The Have Nots and make the current Have Nots a group that has real hope of creating their own brilliant tech or such someday while not being utterly miserable if they aren't currently wealthy.

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#324

Post by neontomo »

Aardwolf wrote: post #198

How comes that the stable diffuse model helps here? Does the fact that it knows what an astronaut on a horse looks like have effect on the audio? Would starting the training from an empty model work too?

I think they only mentioned the horse to illustrate to people that they are using the same tool as what was used to generate those types of images. It's painting a picture for the uninitiated audience. From what I understood, this model is trained on spectrograms instead of horses and the like, resulting in this product.

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#326

Post by primarch »

Could this concept be inverted, to identify music from a text prompt, as in i want a particular vibe and it can go tell me what music fits that description. Ive always thought the ability to find music you like was very lacking, it should not be bounded by a genre instead its usually based on rythmic and melodic structures that appeal to you, regardless of what type of music it might be.

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#327

Post by alvah »

Archipelagia wrote: post #295
halkony wrote: post #290

Earlier quoted context omitted.

I see a lot of AI naysayers neglecting the comparative advantage part. If AI completely eliminates low skill art labour from the job pool, it's not like those affected by it are gonna disintegrate, riot, and restructure society. They have the choice of filling an art niche an AI can't or they can spend that time learning other, more in-demand skills. This also ignores that fact that some companies would rather reallo…

That seems a bit like wishful thinking. People don't have unlimited ability to learn new skills. Training takes time, and someone who spent several years honoring their craft won't be able to pick up a new skill overnight. On top of that, people have preferences regarding their work – even if someone has the ability to do a different work, they might find it less meaningful and less satysfing. Finally, don't ignore t…

Agree 100%. When I was young and idealistic I believed in the "learn new skills" mantra, but learning completely new skills would look a lot different at 50 than it did at 20. When career choices were being made 30 years ago it would have been hard to predict the current & upcoming AI-driven destruction of lower-end "thinking" jobs. Attempting to retrain after ~30 years puts you at a massive disadvantage vs a new graduate (I mentor some of our companies graduates & trainees & I've been assigned a guy in his mid-40s, after a few months I just don't see how he'll get to a point where he's adding value). Not really a personal whinge, as my skillset isn't under immediate threat from any AI I've heard about, though the rate of change in the field is something to behold.

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#328

Post by marktolson »

munificent wrote: post #254
bheadmaster wrote: post #19

This is a genius idea. Using an already-existing and well-performing image model, and just encoding input/output as a spectrogram... It's elegant, it's obvious in retrospective, it's just pure genius. I can't wait to hear some serious AI music-making a few years from now.

As someone who loves making music and loves listening to music made by other humans with intention, it just makes me sad. Sure, AI can do lots of things well. But would you rather live in a world where humans get to do things they love (and are able to afford a comfortable life while doing so) or a world where machines do the things humans love and humans are relegated to the remaining tasks that machines happened to…

I play the piano (badly). There are many other people who can play much better than I. There are simple computer programs which can play better. It doesn't stop me from enjoying it or playing it. Computers have been beating people at Chess for years yet you still see people everywhere enjoying the game. At some point computers will be better than humans at absolutely everything but it shouldn't stop you as a human from enjoying anything.

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#329

Post by tltimeline2 »

adamsmith143 wrote: post #186
pmontra wrote: post #183

A musician friend of mine told me that this is (I freely translate) a perversion, building in frequency and returning time. Don't shoot the messenger. Personally I like the results. I'm totally untrained and couldn't hear any of the issues many comments are pointing out. I guess that all of lounge/elevator music and probably most ad jingles will be automated soon, if automation cost less than human authors.

"Horse Carriage Driver says horseless carriages are abominations. More at 12!"

adamsmith143, commenter. Witness here the apathy of someone unaffected by the suffering of millions and uncommitted to the soul of humanity. DIAF.

Re: Riffusion – Stable Diffusion fine-tuned to generate music

#330

Post by foobarbecue »

xtracto wrote: post #66

This looks great and the idea is amazing. I tried with the prompt: "speed metal" and "speed metal with guitar riffs" and got some smooth rock-balad type music. I guess there was no heavy metal in the learning samples haha. Great work!

Yeah... I just wanted some metal and there's no way to get anything gruff or agressive out of this at all :-(
Post reply on HN