When discussing this with my friends, an interesting question came up: Who owns the music this produces?
Couldn't one generate music and upload that to Spotify and get paid based off the number of listens?
MuseNet
141–150 of 189 posts
Re: MuseNet
#142Impressive. Would this model benefit from something like "dilated attention"? Instead of feeding it raw sound samples, we could split the input into 16 sec, 8 sec, 4 sec and so on slices, assign each slice a "sound vector" serving as a short description of that slice and let the generator take those sound vectors as input. This should supposedly let it gain global consistency in output.
Now an unpopular opinion. I'm not an ML expert, so take my words with reasonable skepticism. This fancy GPT2 model diagram can impress an uninitiated, but we are initiated, right? There is really no science there and it's still the good old numbers grinder: an input of fixed size is passed thru a big random pile of matrix multiplications and sigmoids and yields a fixed size output. We could technically replace this nice looking GTP2 model with a flat stack of matmuls and tanhs, with a ton of weights and given enough powerful GPUs (that would cost tens of millions), train that model and get the same result. It just won't make an impression of science. How are these GTP2 models designed? By somewhat random experiments with the model structure. The key here is the GPU datacenter that could quickly evaluate the model on a huge dataset. The breakthru would be achieving the same quality with very little weights.
Re: MuseNet
#143Earlier quoted context omitted.
It's probably only a matter of time before we have a GauGAN like interface for synthetic music creation...so you could say 'i want a sad song with a soft intro and a buildup of tension here with lyrics covering these emotions and things which lasts 7 minutes'. ML/DL is coming for a lot of the grunt work. It's coming for us as programmers as well. It's probably a few years away, but ML/DL
Given how easy it is to train a Transformer on any sequence data, and given how plentiful open source code is, I'd say "CodeNet" is probably less than a year away. OpenAI will probably do it first given they already have the setup.
That CodeNet would be the SkyNet, essentially. What's shown here looks impressive, but it's the same good old text generator that can produce something that looks very similar to the dataset used to train it. It can't go beyond the dataset and generate something new. From the mathematical point of view, that generator interpolates samples from the dataset and generates a new sample.
To give an idea how big is the gap between MuseNet and CodeNet, we can consider a simple problem of reversing a sequence: [1,2,3,4,5] should become [5,4,3,2,1] and so on. How many samples do you need to look at to understand how to reverse an arbitrary sequence of numbers? Do you need to retrain your brain to reverse a sequence of pictures? No, because instead of memorizing the given samples, you looked at a few and built a mental model of "reversing a sequence of things". Now, the state of the art ML models can reverse sequences as long as they are using the same numbers as in the dataset, i.e. we can train them to reverse any sequence of 1..5 or 1..50 numbers, but once we add 6 to the input, the model instantly fails, no matter how complex and fancy it is. I don't even dare to add a letter to the input. Reason? 6 isn't in the samples it's learnt to interpolate. And CodeNet is supposed to generate a C++ program that would reverse any sequence, btw.
At the moment, ML is kinda stuck at this pictures interpolation stage. For AI, we don't need to interpolate samples, but need to build a "mental model" of what these samples are and as far as I know, we have no clue how to even approach this problem.
Re: MuseNet
#144I don't know the other genres well enough to evaluate them, but the bluegrass one ( https://soundcloud.com/openai_audio/genre-bluegrass ) is pretty bizarre: * Who uses piano as the lead instrument in bluegrass? * They're only using one one note velocity for the whole piece, which misses a huge wealth of variation through rhythmic accent. * Timing generally feels a bit robotic? * The best parts it sounds kind of ok bu…
It's midi. It's meant to be notation, not a realistic composition. I suppose the background "machine gun" piano notes would be a bassy synth combined with 90s electronic music drums. So please do not focus that much on the audio or performance part of it, but on the composition part of it. I do agree they could have trained it about the importance of velocity though. (That neural net, and most young music students ou…
>It's midi. It's meant to be notation, not a realistic composition.
MIDI certainly can represent realistic compositions, when used with good synths, though I agree it's not what it's known for!
> I suppose the background "machine gun" piano notes would be a bassy synth combined with 90s electronic music drums.
Uh, what sort of bluegrass music do you listen to and where can I find some of it?
Re: MuseNet
#145Something I'm curious about: If I make some music I really like through this tool, do I own the copyright to that? Can I turn generated music into an album and sell it? I'm not sure if the site does caching but if it does and me and another person generate the same music, do we both own rights to that?
I got this question a lot about my StyleGAN anime faces & GPT-2-small poetry: https://www.gwern.net/Faces#faq The legal consensus, such as it is, seems to be that (if you did not otherwise agree to a contract/license modifying this in arbitrary ways) you create a new copyright & own it if you use their music-editing tool to tweak settings until you got something you liked, because you are exercising creative control,…
How do you prove that too? Maybe in the future somebody creates a random painting in FuturePaintGAN(TM)+3DCanvasPrinter(TM) that moves millions of people to tears and sells for hundreds of thousands in some auction house. Is that their IP?
What if that person is a monkey[1]? Is it "animal-made art"[2]?
[1] https://en.wikipedia.org/wiki/Pierre_Brassau [2] https://en.wikipedia.org/wiki/Animal-made_art
Re: MuseNet
#146This seems incredibly applicable to musical scores in movies. I can imagine a product where the editor/director/someone inputs a handful of variables (mood, genre, instruments, etc) and timing requests (crescendo beginning at 30s and ending at 75s, calm period from 90s - 120s, etc) and out comes a musical score for the movie that matches up with their scene editing.
Maybe more interesting for video games, where you can have a realtime input of gameplay variables.
Something similar was done in the Left 4 Dead series with the AI Director:
Tim Larkin: We took several steps to keep the music interesting enough that the players would be inclined to keep it on as they play. We keep it changing so it won't become tedious; to this end, we created a music director that runs alongside the AI director, tracking the player's experience rather than their emotional state. We keep the music appropriate to each player's situation and highly personalized. The music engine in Left 4 Dead has a complete client-side, multi-track system per player that is completely unique to that player and can even be monitored by the spectators. Since some of the fun of Left 4 Dead is watching your friends when you're dead, we thought it was important to hear their personal soundtrack as well. This feature is unique to Left 4 Dead.
For single-player it works really well for building tension leading up to being attacked by waves of zombies and creating calm spots after high-stress encounters.
In the online versus mode however players got really good at using the musical hints as "tells" for when certain actions happen in the game. Most notably there were musical signatures consisting of a few notes that would play when the special infected characters spawn. Coordinated teams could use this to their advantage know when the infected team is about to stage an attack (like an ambush at a choke point in the map).
Re: MuseNet
#147Earlier quoted context omitted.
Given how easy it is to train a Transformer on any sequence data, and given how plentiful open source code is, I'd say "CodeNet" is probably less than a year away. OpenAI will probably do it first given they already have the setup.
That CodeNet would be the SkyNet, essentially. What's shown here looks impressive, but it's the same good old text generator that can produce something that looks very similar to the dataset used to train it. It can't go beyond the dataset and generate something new. From the mathematical point of view, that generator interpolates samples from the dataset and generates a new sample. To give an idea how big is the gap…
Yeah, I know what you are saying... But let's just let somebody try this experiment (and somebody eventually will), and we can judge what can or cannot be learned by the results.
We will definitely get a great code autocompleter at the very least..
Re: MuseNet
#148Earlier quoted context omitted.
I got this question a lot about my StyleGAN anime faces & GPT-2-small poetry: https://www.gwern.net/Faces#faq The legal consensus, such as it is, seems to be that (if you did not otherwise agree to a contract/license modifying this in arbitrary ways) you create a new copyright & own it if you use their music-editing tool to tweak settings until you got something you liked, because you are exercising creative control,…
How do you prove that too? Maybe in the future somebody creates a random painting in FuturePaintGAN(TM)+3DCanvasPrinter(TM) that moves millions of people to tears and sells for hundreds of thousands in some auction house. Is that their IP? What if that person is a monkey[1]? Is it "animal-made art"[2]? [1] https://en.wikipedia.org/wiki/Pierre_Brassau [2] https://en.wikipedia.org/wiki/Animal-made_art
How do you prove anything to a court about who owns a copyright? You present what evidence you have and marshal what arguments you can, and hope that the person in the right prevails.
As your own links indicate, animals have no more copyrights any more than a computer program would because they are not human, and copyright is explicitly granted to human creative efforts.
Re: MuseNet
#149Earlier quoted context omitted.
Given how easy it is to train a Transformer on any sequence data, and given how plentiful open source code is, I'd say "CodeNet" is probably less than a year away. OpenAI will probably do it first given they already have the setup.
Can you explain? I'm not an expert on ML by any stretch of the imagination, but you'd think with the sort of stringent logical coherence required to construct useful programs, it'd be a pretty subpar use case. Or do you mean smaller-scope tools to aid programming, like linters and autocompleters?
I wonder if you could find a representation for computer programs that eliminated all of the degrees of freedom that were syntax errors, leaving only valid programs. In a sense that's what an AST is but you can still have invalid ASTs. I bet it would be a lot easier to generate interesting programs in a representation like that.
Re: MuseNet
#150Earlier quoted context omitted.
Given how easy it is to train a Transformer on any sequence data, and given how plentiful open source code is, I'd say "CodeNet" is probably less than a year away. OpenAI will probably do it first given they already have the setup.
I'm working on this. I've been training on Stack Overflow and the model has already learned the syntaxes and common coding conventions of a bunch of different languages all on its own. Excited to see what else it's able to do as I keep experimenting. Some sample outputs (you'll probably want to browse to some of the "Random" questions because by default it's showing "answers" right now and I haven't trained that mode…
These are actually a lot of fun to read. Kudos!