Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open source implementation of Google's MusicLM in PyTorch

github.com

Post reply on HN
Print view View original

11–20 of 23 posts

Re: Open source implementation of Google's MusicLM in PyTorch

#11

Post by albertzeyer »

Lucasoato wrote: post #8

Does anyone know if these models can output also Midi instead of plain audio?

This model is designed to output raw audio.

However, there are many models which do output midi. That's actually much simpler, and has been done already a few years ago.

I thought OpenAI did this. But then, I might misremember, because their Jukebox actually also seems to produce raw audio (https://openai.com/blog/jukebox/).

Edit: Ah, it was even earlier, OpenAI MuseNet, this: https://openai.com/blog/musenet/

However, midi generation is so easy, you even find it in some tutorials: https://www.tensorflow.org/tutorials/audio/music_generation

Re: Open source implementation of Google's MusicLM in PyTorch

#12

Post by swyx »

bevenky wrote: post #2

Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch. https://github.com/lucidrains/musiclm-pytorch/blob/main/musi...

pardon my ignorance - what exactly is involved in reimplementing these models?

i assume there's only a superficial description of the architecture, and no weights to load in, so you'll have to train everything from scratch? do we even have their dataset?

Re: Open source implementation of Google's MusicLM in PyTorch

#13

Post by albertzeyer »

This guy (Phil Wang, https://github.com/lucidrains) seems to have the hobby to just implement all models and papers he finds interesting. See his GitHub page. He has 228 repos, and most of them are some implementation of some machine learning paper. Some of those repos are quite popular.

Re: Open source implementation of Google's MusicLM in PyTorch

#14

Post by f_devd »

swyx wrote: post #12
bevenky wrote: post #2

Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch. https://github.com/lucidrains/musiclm-pytorch/blob/main/musi...

pardon my ignorance - what exactly is involved in reimplementing these models? i assume there's only a superficial description of the architecture, and no weights to load in, so you'll have to train everything from scratch? do we even have their dataset?

Generally it's without weights, but MusicLM is also a WIP. More mature implementations have descriptions on how to train them and follow ups on small scale/crowd-sourced experiments & research[1].

[1]: https://github.com/lucidrains/denoising-diffusion-pytorch

Re: Open source implementation of Google's MusicLM in PyTorch

#16

Post by jamessb »

albertzeyer wrote: post #13

This guy (Phil Wang, https://github.com/lucidrains ) seems to have the hobby to just implement all models and papers he finds interesting. See his GitHub page. He has 228 repos, and most of them are some implementation of some machine learning paper. Some of those repos are quite popular.

The project README thanks "Stability.ai for the generous sponsorship to work and open source cutting edge artificial intelligence research", so it's not necessarily just a hobby (though it's possible they just provide compute resources).

Phil's homepage [1] links to a form [2] where you can suggest a paper for him to implement.

[1]: https://lucidrains.github.io/

[2]: https://forms.gle/Dtrxc6CceHEcqS6X6

Re: Open source implementation of Google's MusicLM in PyTorch

#17

Post by PartiallyTyped »

albertzeyer wrote: post #13

This guy (Phil Wang, https://github.com/lucidrains ) seems to have the hobby to just implement all models and papers he finds interesting. See his GitHub page. He has 228 repos, and most of them are some implementation of some machine learning paper. Some of those repos are quite popular.

The implementation is quite clean too, and provided that you have read the papers, they are easy to understand.

Re: Open source implementation of Google's MusicLM in PyTorch

#18

Post by asciii »

albertzeyer wrote: post #13

This guy (Phil Wang, https://github.com/lucidrains ) seems to have the hobby to just implement all models and papers he finds interesting. See his GitHub page. He has 228 repos, and most of them are some implementation of some machine learning paper. Some of those repos are quite popular.

He is open to consultation and work so the repo is a nice gallery of what's possible and learnable material

https://lucidrains.github.io/

He also is creator of ThisPersonDoesNotExist.com

Re: Open source implementation of Google's MusicLM in PyTorch

#19

Post by alexmolas »

I don't understand how this got so many upvotes. It takes only one minute to read the code and realize that the model is not yet completely implemented. Sometimes I have the feeling that people upvote posts without even reading them...

Of course, it's good work, and knowing lucidrains trajectory it's probably going to be implemented in the following days/weeks. But I wonder how many people have at least opened the link before upvoting it.

Re: Open source implementation of Google's MusicLM in PyTorch

#20

Post by dimatura »

jtode wrote: post #15
kavalg wrote: post #7

Google's MusicLM sounds plausible, but quite dull and even sometimes irritating to my musician's ear.

As another musician, I'll point out that there was a point when the only thing AI could produce visually was a lot of dog faces.

My day job is in ML, but I also enjoy music making as a hobby (on a very amateur level - mostly making 4-bar loops on a handheld tracker or knob twiddling on 90s synths I couldn't afford as a kid). I see an interesting mix of curiosity, hostility and head-in-the-sand attitudes from the musician communities. Though the "head-in-the-sand" component will almost certainly start becoming less prevalent with this and other models that are sure to come out in short order.

I'm pretty sure soon enough we'll start seeing the same kind of dynamics that have played out for the arts community in music, not that the dust has settled there yet. I hope there isn't much negative financial impact on people's livelihoods, but maybe some will be unavoidable. And of course, AI is also coming for programmer's jobs, which will hit even closer to home. The next decade will be "interesting", so to speak.

Post reply on HN