MusicLM: Generating Music from Text • Hacker News Board

Re: MusicLM: Generating Music from Text

#11

Post by notahacker » Fri, Jan 27, 2023, 11:50 AM UTC

survirtual wrote: post #8

This is unbelievable. I would have never expected AI to take the jobs of creatives first, but here we are. Artists and musicians can create much better work than AI, but the AI creates consistent works usable for people who aren’t artists, and need assets for their projects — whether video games, prototyping, storyboarding, etc. It is going to be interesting to see artists revert to being a no-revenue passion again i…

> It is going to be interesting to see artists revert to being a no-revenue passion again instead of being well compensated.

Huh? Music already is a near-zero revenue passion for the vast majority of artists, has been since mainstream access to recorded music if not even earlier, and the well compensated minority's superstar status, royalties for hit songs and sellout tours aren't really threatened by DIY tools that generate nice-sounding, genre-appropriate noise a bit more efficiently than a DAW and sample pack.

Re: MusicLM: Generating Music from Text

#12

Post by pclmulqdq » Fri, Jan 27, 2023, 12:32 PM UTC

nprateem wrote: post #5

Wow that is awesome. Hopefully the first feature they'll add will be to spit out midi rather than wavs (or ML that can decompose wavs to midi)

That's a very hard thing for them to add given the way the model is trained. It looks more like a 1-D version of an image generator than past attempts at music AI, which generated MIDI.

Re: MusicLM: Generating Music from Text

#13

Post by bulbosaur123 » Fri, Jan 27, 2023, 12:38 PM UTC

I'm really excited to hear this has been released so soon after Riffusion.

Perhaps both ycombinator threads on MusicLM can be merged together.

If not, I will just post my comments here. I've always seen music as the "final frontier". With images and text you can afford tiny errors and noise that isn't that noticeable by the human eye, but with music everything has to be impeccable and if a note is slightly off, you can instantly tell. Machine learning models that will be able to create outstanding music, imo, will mark the end-game of creative AI endeavors.

Anyone want to share their experiments with MusicLM, feel free to join community-fan discord and subreddit:

https://reddit.com/r/MusicLM/

https://discord.gg/pjVcsyfCJR

BTW, I really hope this gets integrated in software like Ableton similarly how image generators get slowly integrated into Adobe Creative Suite.

Re: MusicLM: Generating Music from Text

#14

Post by rany_ » Fri, Jan 27, 2023, 12:58 PM UTC

It's just disappointing that Google has a ton of cool AI projects that have nice announcement pages and seem promising but are never available to the general public.

Re: MusicLM: Generating Music from Text

#15

Post by zozbot234 » Fri, Jan 27, 2023, 1:25 PM UTC

notahacker wrote: post #9

nprateem wrote: post #5

Wow that is awesome. Hopefully the first feature they'll add will be to spit out midi rather than wavs (or ML that can decompose wavs to midi)

I'm surprised that ML-based MIDI generation hasn't been done quite a bit earlier and pretty well tbh. Sound is incredibly complex, but sequences of chords and notes aligned with music theory and genre conventions have well defined and easily imitated patterns. I guess part of the reason MIDI generation hasn't been a major research focus is that toy scripts get you a lot of the way. These are pretty good, even the voc…

The old Biaxial-RNN by Daniel D. Johnson generates very good output for MIDI music, albeit limited to a single keyboard-like instrument. It's available at https://github.com/danieldjohnson/biaxial-rnn-music-composit... and AIUI there's a GitHub fork that forward-ports it to up-to-date versions of Python (3.x series) and Theano.

Transformer models are quite a bit more computationally intensive than the LSTM this used, and GPT adds attention mechanisms; but the basic approach is loosely comparable and the LSTM model can be easily trained on a single machine.

Re: MusicLM: Generating Music from Text

#16

Post by Juliate » Fri, Jan 27, 2023, 2:05 PM UTC

survirtual wrote: post #8

This is unbelievable. I would have never expected AI to take the jobs of creatives first, but here we are. Artists and musicians can create much better work than AI, but the AI creates consistent works usable for people who aren’t artists, and need assets for their projects — whether video games, prototyping, storyboarding, etc. It is going to be interesting to see artists revert to being a no-revenue passion again i…

> the AI creates consistent works usable for people who aren’t artists, and need assets for their projects

Right but those will become more and more available, dull, and not original (because feeding from the same « repository »).

So to differentiate and attract/inform/delight/surprise/tell better, they might still have to go to specialized people. Or find a way for those models to, like, hallucinate in different ways.

Virtually anyone today can make their own jewels or even setup a shop on Etsy. At low cost, low price.

Still, there’s a huge market for high quality and original/custom-made jewels. Especially when you want something with a story within it. And kind of a human relationship to the provider too.

Re: MusicLM: Generating Music from Text

#17

Post by gavinray » Fri, Jan 27, 2023, 2:07 PM UTC

zozbot234 wrote: post #15

notahacker wrote: post #9

Earlier quoted context omitted.

I'm surprised that ML-based MIDI generation hasn't been done quite a bit earlier and pretty well tbh. Sound is incredibly complex, but sequences of chords and notes aligned with music theory and genre conventions have well defined and easily imitated patterns. I guess part of the reason MIDI generation hasn't been a major research focus is that toy scripts get you a lot of the way. These are pretty good, even the voc…

The old Biaxial-RNN by Daniel D. Johnson generates very good output for MIDI music, albeit limited to a single keyboard-like instrument. It's available at https://github.com/danieldjohnson/biaxial-rnn-music-composit... and AIUI there's a GitHub fork that forward-ports it to up-to-date versions of Python (3.x series) and Theano. Transformer models are quite a bit more computationally intensive than the LSTM this used,…

Know of anything like this that can do EDM/House music?

Re: MusicLM: Generating Music from Text

#18

Post by flashgordon » Fri, Jan 27, 2023, 2:24 PM UTC

survirtual wrote: post #8

This is unbelievable. I would have never expected AI to take the jobs of creatives first, but here we are. Artists and musicians can create much better work than AI, but the AI creates consistent works usable for people who aren’t artists, and need assets for their projects — whether video games, prototyping, storyboarding, etc. It is going to be interesting to see artists revert to being a no-revenue passion again i…

Yeah this is pretty sad. I play an instrument (used to hate music as a kid but somehow paths crossed). I always felt life is best to be about multiple passions instead of all in one thing (whether work or art or sport etc). This could be a great thing for "indie" hobbyists? Sure use AI for stock things but May be we will value the "local" artist more? Or chamber concerts? Or the good old fashioned private dinner with that author? Or a soccer class with Messi's old teacher/coach who lives down the road.

The teacher in me is also optimistic. May be all this AI generated will be great for teaching and democratizing art for students who cannot afford to learn or have easy access to learn!

Re: MusicLM: Generating Music from Text

#19

Post by sharperguy » Fri, Jan 27, 2023, 2:35 PM UTC

survirtual wrote: post #8

This is unbelievable. I would have never expected AI to take the jobs of creatives first, but here we are. Artists and musicians can create much better work than AI, but the AI creates consistent works usable for people who aren’t artists, and need assets for their projects — whether video games, prototyping, storyboarding, etc. It is going to be interesting to see artists revert to being a no-revenue passion again i…

I like to think about the things which this kind of tech makes possible that wouldn't have been possible at all before. E.g:

> Music generated on the fly for any given scenario, that is unique every time.

> Generating custom royalty-free samples which can be used by actual artists as part of a greater work.

> A way to learn about musical styles, motifs and timbres by creating many custom examples

> A way to experiment by generated many hundreds of examples, clashing together many different types of sounds together to find inspiration

Re: MusicLM: Generating Music from Text

#20

Post by college_physics » Fri, Jan 27, 2023, 2:46 PM UTC

Art in all its forms had always serious problems getting funded but things only seem to get worse at each iteration. In older times the problem was that most people had other worries, so artist numbers were small and they seeked patronage from the elites that could afford it.

Later we had mass commercialization which "democratized" art but artists got squeezed by the intermediaries / distributors that enabled the mass market. As digital mass reproduction became a thing, this ushered zero marginal cost dynamics which is essentially still an unresolved stalemate.

The "AI" generation of all sorts of combinatoric works without attribution or payment comes to compound the challenges. You now have arbitrary amounts of derivative art that can be reproduced at zero cost.

Never a dull moment in homo sapiens land.