Open source implementation of Google's MusicLM in PyTorch • Hacker News Board

Re: Open source implementation of Google's MusicLM in PyTorch

#21

Post by dimatura » Tue, Jan 31, 2023, 3:07 PM UTC

kolinko wrote: post #9

Lucasoato wrote: post #8

Does anyone know if these models can output also Midi instead of plain audio?

Not out of the box, afaik. They produce spectograms that get converted into wav/mp3.

I think that description applies to Riffusion, one of the earlier models in this area that was a pretty straightforward to adapt image-based diffusion models to making music, since you can treat spectrograms as images. But this model uses "soundstream", which is another model that has its own paper. It's described as a "neural audio codec" which, by itself, is a model that encodes and decodes audio into "tokens"; so sort of like other codecs (eg, MP3) except that the compressed representation it uses is a more high-level learned representation. This model outputs the tokens which are then decoded by soundstream. The tokens probably encode a lot of the same kind of spectral information contained in spectrograms (or similarly, mel-frequency features) but seem to be a little bit more expressive/data efficient.

Re: Open source implementation of Google's MusicLM in PyTorch

#22

Post by jtode » Wed, Feb 01, 2023, 9:57 AM UTC

dimatura wrote: post #20

jtode wrote: post #15

Earlier quoted context omitted.

As another musician, I'll point out that there was a point when the only thing AI could produce visually was a lot of dog faces.

My day job is in ML, but I also enjoy music making as a hobby (on a very amateur level - mostly making 4-bar loops on a handheld tracker or knob twiddling on 90s synths I couldn't afford as a kid). I see an interesting mix of curiosity, hostility and head-in-the-sand attitudes from the musician communities. Though the "head-in-the-sand" component will almost certainly start becoming less prevalent with this and other…

I also think that it's still very new, and we have not yet discovered the creative limitations of the tech.

I personally expect that it will turn out that there is zero creativity inherent to the tech, and that it will become apparent after a while that without constant new training input from real brains, the output will not creatively evolve, and therefore become boring to many and end up relegated to elevators, hold music, etc, while people rebel and quite possibly have another folk music explosion. Banjos refuse to die, and that will remain true.

I might be wrong, but that's how I'd bet my twenty bucks.

Re: Open source implementation of Google's MusicLM in PyTorch

#23

Post by kavalg » Wed, Feb 01, 2023, 1:58 PM UTC

jtode wrote: post #15

kavalg wrote: post #7

Google's MusicLM sounds plausible, but quite dull and even sometimes irritating to my musician's ear.

As another musician, I'll point out that there was a point when the only thing AI could produce visually was a lot of dog faces.

I am not arguing that it won't be successful, just that I don't like it at its current stage of progress. Actually most of the music that people hear our days is nothing special either (either quite dumb or just a rinse-repeat of some older successful musical forms).