Open source implementation of Google's MusicLM in PyTorch
1–10 of 23 posts
Re: Open source implementation of Google's MusicLM in PyTorch
#2Implementation of MusicLM, Google's new SOTA model for music generation using attention networks, in Pytorch.
https://github.com/lucidrains/musiclm-pytorch/blob/main/musi...
Re: Open source implementation of Google's MusicLM in PyTorch
#3This question is a tangent to your work. Having never used music LMs, and only being cursorily aware of them - how do you keep up with the sota in your field?
Re: Open source implementation of Google's MusicLM in PyTorch
#4Seems to be an early WIP.
Re: Open source implementation of Google's MusicLM in PyTorch
#5Seems to be an early WIP.
This one is AudioLM modified from here https://github.com/lucidrains/audiolm-pytorch to support the music generation needs of Mulan.
Re: Open source implementation of Google's MusicLM in PyTorch
#6Nice work!
Won't the model training be a lot of cost to bear, though?
Re: Open source implementation of Google's MusicLM in PyTorch
#7Google's MusicLM sounds plausible, but quite dull and even sometimes irritating to my musician's ear.
Re: Open source implementation of Google's MusicLM in PyTorch
#8Does anyone know if these models can output also Midi instead of plain audio?
Re: Open source implementation of Google's MusicLM in PyTorch
#9Does anyone know if these models can output also Midi instead of plain audio?
Not out of the box, afaik. They produce spectograms that get converted into wav/mp3.
Re: Open source implementation of Google's MusicLM in PyTorch
#10Does anyone know if these models can output also Midi instead of plain audio?
No. They can’t.
You could train a model that could, but these models can’t.
Paper: https://google-research.github.io/seanet/musiclm/examples/
Quote: “By relying on pretrained and frozen MuLan, we need audio- only data for training the other components of MusicLM. We train SoundStream and w2v-BERT on the Free Music Archive (FMA) dataset (Defferrard et al., 2017), whereas the tokenizers and the autoregressive models for the seman- tic and acoustic modeling stages are trained on a dataset con- taining five million audio clips, amounting to 280k hours of music at 24 kHz.”
Tldr: you can only get out of these models what you put in, and these ones are trained on raw audio.
If you want midi output, you need to train a model on midi data.