I'm not sure what I should be impressed by. Maybe there's some real technical feat happening here, but I feel like a basic mad-libs style algorithm could produce something better.
Artificial Intelligence Generates Christmas Song from Holiday Image
71–80 of 115 posts
Re: Artificial Intelligence Generates Christmas Song from Holiday Image
#72I'm not sure what I should be impressed by. Maybe there's some real technical feat happening here, but I feel like a basic mad-libs style algorithm could produce something better.
I'm not very familiar with mad-libs so correct me if I was wrong. I think generating a lyrics passage (zero hard-coded rule on content or grammar or anything) from an image would not be something you can do with mad-libs.
Re: Artificial Intelligence Generates Christmas Song from Holiday Image
#73One of the cofounders of the Echonest (acquired by Spotify) created this back in 2004: "A Singular Christmas" was composed and rendered in 2004. It is the automatic statistical distillation of hundreds of Christmas songs; the 16-song answer to the question asked of a bank of computers: "What is Christmas Music, really?" https://soundcloud.com/bwhitman/sets/a-singular-christmas
Very interesting reference! Deep learning is statistical, so this is sorta one of the spiritual predecessors.
Re: Artificial Intelligence Generates Christmas Song from Holiday Image
#74Earlier quoted context omitted.
I mean, where does the Christmas element comes from? The image alone, the music it was trained with, or is it somehow hardcoded in the algorithm?
The Christmas element comes from 1. the image, and 2. a 4800-dimensional RNN sentence encoding bias generated from ~30 Christmas songs. Not sure how to hardcode this.
I'm interested that you used some Christmas songs as training (which wasn't obvious from what I read of the paper). Were they pop songs, traditional, or a mix?
Further to my comment up there[0] - and I don't wish to sound a grinch because this is a really cool project - but would I be right in thinking you spent more time on the image description than the music?
I saw that you specify a scale for the melody, would it be either possible to use a mode to generate the accompaniment around, so that the melody can move diatonically and risk too many clashes, or to allow the melody to follow the chord sequence somehow?
Again, sorry if I sound too critical. It's a really awesome thing you've done, and I'm just a guy that listens to the music instead of the lyrics.
Re: Artificial Intelligence Generates Christmas Song from Holiday Image
#75Re: Artificial Intelligence Generates Christmas Song from Holiday Image
#76"The best Christmas present in the world is a blessing." This algorithm is throwing down some wisdom
I think generating fortune cookies is a really low hanging fruit for current AI. Someone could put it together in a week-end.
I am the main developer. You are literally right on the time spent...
Our original focus was writing a research paper on hierarchical music generation. Composing a song from an image is just one of the "fun applications" that we spent a little time on, to promote the interestingness of our method. I started Saturday afternoon, and was basically done by Sunday night.
Re: Artificial Intelligence Generates Christmas Song from Holiday Image
#77I am pretty amazed at the effort that nVidia is putting into its corporate rebranding effort. I wonder if, in the not too distant future, they will be the AI company that also makes Graphics cards sometimes.
The other thing I find really amazing about it, coming from IBM, is that IBM has invested a ton of money in IBM Watson but they sold off their foundry business (could have made massively parallel AI machines) and their systems business is a fraction of what it was.
Looking at what can be done when you're leading versus when you are following is really sobering to me.
Re: Artificial Intelligence Generates Christmas Song from Holiday Image
#78Earlier quoted context omitted.
The Christmas element comes from 1. the image, and 2. a 4800-dimensional RNN sentence encoding bias generated from ~30 Christmas songs. Not sure how to hardcode this.
I'm interested that you used some Christmas songs as training (which wasn't obvious from what I read of the paper). Were they pop songs, traditional, or a mix? Further to my comment up there[0] - and I don't wish to sound a grinch because this is a really cool project - but would I be right in thinking you spent more time on the image description than the music? I saw that you specify a scale for the melody, would it…
Thanks for the comments! Are you asking the lyrics or music generation?
For lyrics, we actually didn't train on Christmas songs. Training data was a large collection of romance novels. (See neural-storyteller by Jamie Kiros). The "Christmas trick" we did was applying a "style shifting" after image captioning and before lyrics generation, where the shifting vector was obtained from ~30 Christmas songs.
For the music generation. Although we are aware of some basic music performing rules, such as melody following chord etc, we actually didn't add this kind of rules.
For the blues scale here's the thing. I didn't really know much about music, so I spent several hours reading things like basicmusictheory.com. It happened to introduce blues so we just used it. But you're right on the relevance between blues and pop: only a very small percentage in our pop music collection is blues, after we ran the scale-checking code.
Re: Artificial Intelligence Generates Christmas Song from Holiday Image
#79Earlier quoted context omitted.
I think generating fortune cookies is a really low hanging fruit for current AI. Someone could put it together in a week-end.
I am the main developer. You are literally right on the time spent... Our original focus was writing a research paper on hierarchical music generation. Composing a song from an image is just one of the "fun applications" that we spent a little time on, to promote the interestingness of our method. I started Saturday afternoon, and was basically done by Sunday night.
Well, your project is quite a bit more complex than just making fortune cookie messages. Still amazing how much can be accomplished quickly with current AI tech.
Thumbs up for the cool project.
Re: Artificial Intelligence Generates Christmas Song from Holiday Image
#80One of the cofounders of the Echonest (acquired by Spotify) created this back in 2004: "A Singular Christmas" was composed and rendered in 2004. It is the automatic statistical distillation of hundreds of Christmas songs; the 16-song answer to the question asked of a bank of computers: "What is Christmas Music, really?" https://soundcloud.com/bwhitman/sets/a-singular-christmas
Very interesting reference! Deep learning is statistical, so this is sorta one of the spiritual predecessors.
By my understanding, yes, in the sense that certain parts of each algorithm are looking to minimize something...the former model used principal components analysis, which is linear in the sense that you are using transforms which pick out for the least correlated pieces of a huge chunk of data, whereas neural networks, which uses a combination of linear and non-linear layers picked by the user to minimize, "errors."
What's interesting is that the former model sounds so much, "better." I wonder if anyone could chime in about how our ears and auditory nerves or perhaps auditory cognition works, and whether they are more, "principal component analysis-y" somehow than "error minimization-y" or something relating to the actual math, which may explain why this new neural network christmas song sounds like absolute crap to us, whereas the older version sounds pretty amazing. Also, whether my understanding of the underlying math is correct or not.