I want to do the reverse: pass in some music and have it describe what I'm hearing
https://samplab.com is doing something like that.
MusicLM: Generating music from text
21–30 of 114 posts
Re: MusicLM: Generating music from text
#22Demo: https://google-research.github.io/seanet/musiclm/examples/
MusicCaps dataset: https://www.kaggle.com/datasets/googleai/musiccaps
Re: MusicLM: Generating music from text
#23Weights when though. :-(
Re: MusicLM: Generating music from text
#24I don't really understand why this approached is pushed for music. You can overpaint an image, but you can't do that with a song. Cutting an image to reintroduce coherence is easy too. For a song you need midi, or another symbolic representation. That was the approach of pop2piano (unfortunately it is limited to covers, not generating from scratch). And even if a song generated this is OK, listening to half an hour f…
I understand what you're saying, although it could be argued that at least for some types of image tasks one would prefer something like an SVG output with layers to make it easier to edit.
For music, I think it's partly an academic question of "can we do it" rather than trying to maximize immediate practical usefulness. There's already quite a bit of work on symbolic music generation (mostly MIDI), a lot of it quite competent, especially in more constrained domains like NES chiptunes or classical piano, so a full text-to-audio pipeline probably seemed a more interesting research problem.
And for a lot of use cases, where people might truly not care too much about tweaking the output to their liking, the generated audio might be good enough; the examples were pretty plausible to my ear, if somewhat lo-fi sounding (probably because it's operating at 24kHz, compared to the more standard 44-48kHz).
In the future a more hybrid approach probably makes sense for at least some applications, where MIDI is generated along with some way of specifying the timbre for each instrument (hopefully something better than general MIDI, though even that would be fun; not sure if it's been done). I'm sure that in the near we'll see a lot more work in the DAW and plugin space to have these kind of things built-in, but in a way that they can be edited by the user.
Re: MusicLM: Generating music from text
#25Demo: https://google-research.github.io/seanet/musiclm/examples/
I love that there's a whole section for accordion examples. I think the accordion rap has a bad word in it. Accordion techno works surprisingly well. They really need to bump up to 48 KHz so all the music doesn't sound like it's being played over a telephone. A factor of two in the cost shouldn't be prohibitive. So much of the audio generation stuff I've seen has fatal flaws like this baked into the dataset and/or tr…
The "problem" is copyright, the music industry has way more power than the people that were copyright infringed by Stable Diffusion et al. And probably no artist that makes a living from their music will voluntarely give his music to AI.
Re: MusicLM: Generating music from text
#26I don't really understand why this approached is pushed for music. You can overpaint an image, but you can't do that with a song. Cutting an image to reintroduce coherence is easy too. For a song you need midi, or another symbolic representation. That was the approach of pop2piano (unfortunately it is limited to covers, not generating from scratch). And even if a song generated this is OK, listening to half an hour f…
I understand what you're saying, although it could be argued that at least for some types of image tasks one would prefer something like an SVG output with layers to make it easier to edit. For music, I think it's partly an academic question of "can we do it" rather than trying to maximize immediate practical usefulness. There's already quite a bit of work on symbolic music generation (mostly MIDI), a lot of it quite…
awesome-sheet-music lists a number of sheet music archives
https://github.com/ad-si/awesome-sheet-music
Other libraries of (Royalty Free, Public Domain) sheet music:
Explainable artificial intelligence: https://en.wikipedia.org/wiki/Explainable_artificial_intelli...
FWIU, Current LLMs can't yet do explainable AI well enough to satisfy the optional Attribution clause of e.g. Creative Commons licenses?
"Sufficiently Transformative" is the current general copyright burden according to precedent; Transformative use and fair use: https://en.wikipedia.org/wiki/Transformative_use
Re: MusicLM: Generating music from text
#27Demo: https://google-research.github.io/seanet/musiclm/examples/
I love that there's a whole section for accordion examples. I think the accordion rap has a bad word in it. Accordion techno works surprisingly well. They really need to bump up to 48 KHz so all the music doesn't sound like it's being played over a telephone. A factor of two in the cost shouldn't be prohibitive. So much of the audio generation stuff I've seen has fatal flaws like this baked into the dataset and/or tr…
are there upscalers in the wavenet world like there are in convnets?
Re: MusicLM: Generating music from text
#28Yet another cherry picked attempt at music gen with a "demo" page that only contains the outputs that happen to not sound like incoherent noise. There's a reason ChatGPT and Midjourney are so popular, but no music-gen tools have even come close: you can actually create stuff with them that is useful and/or enjoyable. Good music gen is much harder, and the reasons for this (still unclear) are pretty important to the f…
Well... no all releases have to be toys for your average twitter user to play with. Most of them are just academic stuff to show team progress.
Re: MusicLM: Generating music from text
#29Earlier quoted context omitted.
I love that there's a whole section for accordion examples. I think the accordion rap has a bad word in it. Accordion techno works surprisingly well. They really need to bump up to 48 KHz so all the music doesn't sound like it's being played over a telephone. A factor of two in the cost shouldn't be prohibitive. So much of the audio generation stuff I've seen has fatal flaws like this baked into the dataset and/or tr…
The "problem" is copyright, the music industry has way more power than the people that were copyright infringed by Stable Diffusion et al. And probably no artist that makes a living from their music will voluntarely give his music to AI.
They should be able to get a significant amount of royalty free music. Granted it will generally be of a lower quality and/or be older but it should still be more than significant.
Re: MusicLM: Generating music from text
#30Interesting concept! I wonder how the hinders around copywriting will be solved.