MusicLM: Generating Music from Text
google-research.github.io
MusicLM: Generating Music from Text
1–10 of 46 posts
Re: MusicLM: Generating Music from Text
#2Link to paper: https://arxiv.org/abs/2301.11325 (the one on the site seems broken)
Even with the knowledge that something like this is inevitabely coming, it's pretty crazy to see the examples. The fact it can create fitting music from painting descriptions (including ones such as or Guernica) is unexpected. And it seems to generate almost naturalistic vocals with reasonable-sounding lyrics, with no embedded structure for lyrics or anything.
The paper itself is pretty cool. While GPT-3 was just "GPT-2, but bigger", this seems to have required some clever combination of models. Wonder if similar ideas will be needed for longer-form text or video generation.
Re: MusicLM: Generating Music from Text
#3I can't seem to load any of these WAV files; any mirrors?
Re: MusicLM: Generating Music from Text
#4Ohh I want a product where I can play and experiment with it!
Re: MusicLM: Generating Music from Text
#5Wow that is awesome. Hopefully the first feature they'll add will be to spit out midi rather than wavs (or ML that can decompose wavs to midi)
Re: MusicLM: Generating Music from Text
#6Ohh I want a product where I can play and experiment with it!
yeah, same here. I'm also curious to see what the average quality is like, compared to those hand-picked ones.
Re: MusicLM: Generating Music from Text
#7Link to paper: https://arxiv.org/abs/2301.11325 (the one on the site seems broken) Even with the knowledge that something like this is inevitabely coming, it's pretty crazy to see the examples. The fact it can create fitting music from painting descriptions (including ones such as or Guernica) is unexpected. And it seems to generate almost naturalistic vocals with reasonable-sounding lyrics, with no embedded structur…
[deleted]
Re: MusicLM: Generating Music from Text
#8This is unbelievable. I would have never expected AI to take the jobs of creatives first, but here we are. Artists and musicians can create much better work than AI, but the AI creates consistent works usable for people who aren’t artists, and need assets for their projects — whether video games, prototyping, storyboarding, etc.
It is going to be interesting to see artists revert to being a no-revenue passion again instead of being well compensated. That “well compensated” did not last long for them and I feel for the struggle. At the same time, a part of me feels art has no place being motivated by money anyway. Perhaps this change will restore the balance. Artists will need to get real jobs again like the rest of us and fund their art as a side project.
I hope an open model for music generation will be soon to follow, it will really be a big deal.
Re: MusicLM: Generating Music from Text
#9Wow that is awesome. Hopefully the first feature they'll add will be to spit out midi rather than wavs (or ML that can decompose wavs to midi)
I'm surprised that ML-based MIDI generation hasn't been done quite a bit earlier and pretty well tbh. Sound is incredibly complex, but sequences of chords and notes aligned with music theory and genre conventions have well defined and easily imitated patterns. I guess part of the reason MIDI generation hasn't been a major research focus is that toy scripts get you a lot of the way.
These are pretty good, even the vocals are OK, although I wonder how much if parts of the prompt like "it may be used during a festival during two songs for a buildup" are actually adding much to the mix and the music's association with the painting descriptions is as loose as I'd expect.
Re: MusicLM: Generating Music from Text
#10This is unbelievable. I would have never expected AI to take the jobs of creatives first, but here we are. Artists and musicians can create much better work than AI, but the AI creates consistent works usable for people who aren’t artists, and need assets for their projects — whether video games, prototyping, storyboarding, etc. It is going to be interesting to see artists revert to being a no-revenue passion again i…
> It is going to be interesting to see artists revert to being a no-revenue passion again instead of being well compensated.
Art should be a communication way, a hobby... All the people should have time to do art, do love or whatever they want... Also, all the people, who are able, should do productivity work, to maintain the physical bodies.