Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MusicLM: Generating music from text

arxiv.org

Post reply on HN
Print view View original

11–20 of 114 posts

Re: MusicLM: Generating music from text

#11

Post by robitsT6 »

not-chatgpt wrote: post #10
TaupeRanger wrote: post #8

Yet another cherry picked attempt at music gen with a "demo" page that only contains the outputs that happen to not sound like incoherent noise. There's a reason ChatGPT and Midjourney are so popular, but no music-gen tools have even come close: you can actually create stuff with them that is useful and/or enjoyable. Good music gen is much harder, and the reasons for this (still unclear) are pretty important to the f…

Their current demos seem worse than riffusion results I get on average. Music gen is hard because music is inherently a composed of many many different instruments, each with unique sound and function. Simply training end-to-end will almost always end up badly.

I'm sorry, I like riffusion as much as the next guy, but in what world is any riffusion example better than the literal first example on this demo page?

Re: MusicLM: Generating music from text

#12

Post by not-chatgpt »

robitsT6 wrote: post #11
not-chatgpt wrote: post #10

Earlier quoted context omitted.

Their current demos seem worse than riffusion results I get on average. Music gen is hard because music is inherently a composed of many many different instruments, each with unique sound and function. Simply training end-to-end will almost always end up badly.

I'm sorry, I like riffusion as much as the next guy, but in what world is any riffusion example better than the literal first example on this demo page?

Might be personal preference, but I don't think those examples are good at all. The only thing I am impressed with is the story mode and conditioning. I typically use riffusion to generate swing/electropop music, could be I'm too biased.

Re: MusicLM: Generating music from text

#13

Post by modeless »

georgehill wrote: post #2

Demo: https://google-research.github.io/seanet/musiclm/examples/

I love that there's a whole section for accordion examples. I think the accordion rap has a bad word in it. Accordion techno works surprisingly well.

They really need to bump up to 48 KHz so all the music doesn't sound like it's being played over a telephone. A factor of two in the cost shouldn't be prohibitive. So much of the audio generation stuff I've seen has fatal flaws like this baked into the dataset and/or training process that ensure the output can't sound good even in theory, it's kinda frustrating.

It's also frustrating that nobody AFAIK has trained a music model on an actually large dataset. We're training large language models on a significant fraction of the text of the whole internet! Where are the audio models trained on a significant fraction of all recorded music? This one was trained (in part) on a dataset of 280k hours of music, if I read the paper correctly. I don't think that comes anywhere close to being a significant fraction of all recorded music.

Re: MusicLM: Generating music from text

#15

Post by zone411 »

not-chatgpt wrote: post #10
TaupeRanger wrote: post #8

Yet another cherry picked attempt at music gen with a "demo" page that only contains the outputs that happen to not sound like incoherent noise. There's a reason ChatGPT and Midjourney are so popular, but no music-gen tools have even come close: you can actually create stuff with them that is useful and/or enjoyable. Good music gen is much harder, and the reasons for this (still unclear) are pretty important to the f…

Their current demos seem worse than riffusion results I get on average. Music gen is hard because music is inherently a composed of many many different instruments, each with unique sound and function. Simply training end-to-end will almost always end up badly.

> Music gen is hard because music is inherently a composed of many many different instruments, each with unique sound and function. Simply training end-to-end will almost always end up badly.

Yes! That's what I figured when I started my project to create an AI assistant for melodies (http://melodies.ai/). I'm quite sure that splitting it up is the way to go, and that the first AI hit song will not be created as a full song but by combining instruments/vocals.

Re: MusicLM: Generating music from text

#16

Post by andreyk »

not-chatgpt wrote: post #10
TaupeRanger wrote: post #8

Yet another cherry picked attempt at music gen with a "demo" page that only contains the outputs that happen to not sound like incoherent noise. There's a reason ChatGPT and Midjourney are so popular, but no music-gen tools have even come close: you can actually create stuff with them that is useful and/or enjoyable. Good music gen is much harder, and the reasons for this (still unclear) are pretty important to the f…

Their current demos seem worse than riffusion results I get on average. Music gen is hard because music is inherently a composed of many many different instruments, each with unique sound and function. Simply training end-to-end will almost always end up badly.

FYI, they don't actually train end-to-end, they have three separate models that are each trained via self-supervised learning independently (I think, I haven't read the paper very carefully).

Re: MusicLM: Generating music from text

#18

Post by woolion »

I don't really understand why this approached is pushed for music. You can overpaint an image, but you can't do that with a song. Cutting an image to reintroduce coherence is easy too. For a song you need midi, or another symbolic representation. That was the approach of pop2piano (unfortunately it is limited to covers, not generating from scratch). And even if a song generated this is OK, listening to half an hour full of AI mistakes is really tiring. With a symbolic representation you could at least fix the mistakes if there is one good output.

Re: MusicLM: Generating music from text

#19

Post by jsemrau »

not-chatgpt wrote: post #10
TaupeRanger wrote: post #8

Yet another cherry picked attempt at music gen with a "demo" page that only contains the outputs that happen to not sound like incoherent noise. There's a reason ChatGPT and Midjourney are so popular, but no music-gen tools have even come close: you can actually create stuff with them that is useful and/or enjoyable. Good music gen is much harder, and the reasons for this (still unclear) are pretty important to the f…

Their current demos seem worse than riffusion results I get on average. Music gen is hard because music is inherently a composed of many many different instruments, each with unique sound and function. Simply training end-to-end will almost always end up badly.

Makes one wonder if there are models that rely on loops.
Post reply on HN