Open-sourcing AudioCraft: Generative AI for audio

Post by speedgoose » Wed, Aug 02, 2023, 8:56 PM UTC

peteforde wrote: post #209

I just ran all of the cited installation steps, which appear to have been successful... but I am now experiencing a profound sense of "now what?" There doesn't appear to be any new CLI executables installed, and the documentation links to an API but there's no clues on how to actually process a prompt. What am I missing? Alternatively, I wouldn't mind using it in a Notebook but so far this thread doesn't link to anyt…

    python demos/musicgen_app.py

Post by smoldesu » Wed, Aug 02, 2023, 8:57 PM UTC

__loam wrote: post #210

smoldesu wrote: post #89

Earlier quoted context omitted.

That, or this is an ongoing research lab (FAIR) that has existed for ~half a decade and has advanced the state-of-the-art in AI further than Apple, Microsoft and Google combined.

I would be pretty shocked if meta were that far ahead of all 3 of those companies, all of which are also spending a fuck load on internal AI research.

Post by j_maffe » Wed, Aug 02, 2023, 9:00 PM UTC

JeremyNT wrote: post #99

gavman wrote: post #13

> MusicGen, which was trained with Meta-owned and specifically licensed music, generates music from text-based user inputs, while AudioGen, which was trained on public sound effects, generates audio from text-based user inputs. Meta is really clearly trying to differentiate themselves from OpenAI here. Open source + driving home "we don't use data we haven't paid for / don't own".

The fact that Meta is able to lie and call their restrictive licensing open source is nearly as misleading as "OpenAI." We need to do better than to repeat these claims uncritically. The weight licenses are not "open source" by any useful definition, and we should not give Meta kudos for their misleading PR (especially considering that they almost surely ignored any copyright when training these things - rules for th…

Post by chrisjj » Wed, Aug 02, 2023, 9:01 PM UTC

emporas wrote: post #94

skybrian wrote: post #83

As an amateur musician I’m wondering if there are any of these audio generators that you can give a tune or chord progression to riff on. ABC format maybe? There are lots of folk tunes on thesession.org. Could you generate a rhythm track? Ideally you could make songs one track at a time, by giving it a mix of the previous tracks and asking it to make another track for an instrument. Or, give it a track and ask it to…

If you mean to give a source of melody of 30sec and extend that melody into a full song, yes MusicGen can do that. There are two ways to extend a song based on a melody: 1) give a sample, and continue the song from that sample as close as possible, and 2) give a melody as an inspiration. They both work in varying degrees of success. Audiocraft on github, issues or discussion sections have a lot of questions answered.

Post by mcbits » Wed, Aug 02, 2023, 9:04 PM UTC

cosmojg wrote: post #113

ericpauley wrote: post #43

Earlier quoted context omitted.

My understanding (IANAL) [1] is that copyright licenses have no say on the output of software. Further, CC licenses don't say anything about running or using software (or model weights). It's therefore questionable whether the CC-BY-NC license actually prevents commercial use of the model. [1] https://opensource.stackexchange.com/questions/12070/allowed...

You're correct, but no one has had the balls (or the lawyers) to clarify this in court yet. Expect to see hosting providers complying with takedown requests for the foreseeable future.