What makes Bach sound like Bach? New dataset teaches algorithms classical music
1–10 of 31 posts
Re: What makes Bach sound like Bach? New dataset teaches algorithms classical music
#2This is really interesting, but it confused me at first. Given the title and the problem posed in the article's intro, I figured this would be a dataset of sheet music, ie. the notes and durations specified in some printed music. However, reading more it appears to be focused on recordings (ie. audio) and annotating those recordings with information about where each note starts and stops.
So to me this seems more directly applicable to transcription (ie. taking audio and turning it into sheet music) or synthesis (taking sheet music and turning it into audio of a human-sounding performance) than it does to composition or finishing unfinished works by famous composers. The output of the compositional process is generally sheet music, not audio, so it seems to make more sense that problems around composition would be trained and learn in the sheet music domain.
I'm not a machine learning researcher though! This is just my impression as a musician.
Re: What makes Bach sound like Bach? New dataset teaches algorithms classical music
#3The question of what makes Bach sound like Bach is, needless to say, not addressed.
The actual thing they're reporting is:
'“You need to be able to say from 3 seconds and 50 milliseconds to 78 milliseconds, this instrument is playing an A. But that’s impractical or impossible for even an expert musician to track with that degree of accuracy.”
The UW research team overcame that challenge by applying a technique called dynamic time warping — which aligns similar content happening at different speeds — to classical music performances. This allowed them to synch a real performance, such as Beethoven’s ‘Serioso’ string quartet, to a synthesized version of the same piece that already contained the desired musical notations and scoring in digital form.
Time warping and mapping that digital scoring back onto the original performance yields the precise timing and details of individual notes that make it easier for machine learning algorithms to learn from musical data.'
It also mentions that they attempted to apply existing deep learning algorithms designed for speech recognition to their new dataset, hoping to be able to accomplish a task such as predict a single missing note from a long string of notes. It does not say whether this worked.
Re: What makes Bach sound like Bach? New dataset teaches algorithms classical music
#4This is really interesting, but it confused me at first. Given the title and the problem posed in the article's intro, I figured this would be a dataset of sheet music, ie. the notes and durations specified in some printed music. However, reading more it appears to be focused on recordings (ie. audio) and annotating those recordings with information about where each note starts and stops. So to me this seems more dir…
Hi haberman,
I'm one of the authors on this paper. You're right that the most direct applications of this dataset are transcription and synthesis. One of the cool aspects of end-to-end learning models is that they discover a "representation" of data that can be useful when applied to other tasks. We speculate about some tasks like recommendation and composition on our website:
http://homes.cs.washington.edu/~thickstn/musicnet.html
We're also interested in music like jazz and pop, for which good scores are often unavailable. Classical music is nice for training models because we can use sheet music as labels to learn a representation. Many aspects of this representation, such as rhythm and harmony, may transfer to other musical genres. Learning about classical recordings could bootstrap learning for other kinds of musical audio.
So while you're right that it's probably easier to learn a model to complete Bach using symbolic sheet music, we feel that addressing complex tasks directly from raw audio is worthwhile!
Re: What makes Bach sound like Bach? New dataset teaches algorithms classical music
#5The question of what makes Bach sound like Bach is, needless to say, not addressed. The actual thing they're reporting is: '“You need to be able to say from 3 seconds and 50 milliseconds to 78 milliseconds, this instrument is playing an A. But that’s impractical or impossible for even an expert musician to track with that degree of accuracy.” The UW research team overcame that challenge by applying a technique called…
Hi savanaly,
When we talk about "what makes Bach sound like Bach," the technical concept we have in mind is the recent work in computer vision on style transfer. For example,
https://arxiv.org/abs/1508.06576
We are excited to work on adapting these models to the musical domain!
As for note prediction, you can see our results in our paper:
https://arxiv.org/abs/1611.09827
Our are results for simple (2-layer, not very "deep" models); we were interested in understanding the low-level "features" of music rather than building a model that maximizes performance. Nevertheless, the results are quite promising; I'm confident that someone using our dataset with a deep network and a lot of gpus could blow our numbers out of the water! :)
Tutorials on how to set up and evaluate this task are available on our website:
Re: What makes Bach sound like Bach? New dataset teaches algorithms classical music
#6Albert Schweitzer pointed out (in https://www.amazon.com/dp/0486216314 ) that in many cases it's hard to understand Bach's music without understanding the lyrics. The mood of the music will change from cheerful to somber (or whatever) seemingly randomly, but if you understand the lyrics, it's not random.
Re: What makes Bach sound like Bach? New dataset teaches algorithms classical music
#7i ran into a few issues trying to study classical music with a computer. first of all is merely putting in some representation of the musical score into a file. This was accomplished by MIDI but I am hoping for a more standard way that looks more like the notes of a score.
Another problem is once you have the music there's a tremendous amount of "interpretation" that a musician does. the nodes may each read 1/8 but a musician might add or subtract 1/64 has he/she feels is good.
other times the change is more mathematical 1/8+1/8+1/8 might have to actually be read 1/12+1/12+1/12 = 1/4 but that is much easier to fit into a computer
I have said nothing of dynamics (loud/quiet), articulation (stoccatto, slurring etc).
scores are available in IMSLP and other sources. but are computer files available as well?
Re: What makes Bach sound like Bach? New dataset teaches algorithms classical music
#8This announces a new dataset where recorded performances are precisely synchronized to MIDI transcriptions. Obviously the article doesn't seem to get the implications quite right (it's very useful for performance-related research, not so much for AI composition).
As a composer, the coolest potential I see here is training a model to create realistic mockups from MIDI compositions. For that purpose, though, it would be better to start with a fully monophonic/solo-instrument dataset, which would simplify the learning. Also, MIDI data is not entirely sufficient: annotations on dynamics and playing technique would be necessary to make a good mockup tool, since this is the kind of information one might even give to human performers.
Anyways, it would be tough for such a tool to catch up with current state-of-the-art, sample-based mockup tools, which are already baffling in their realism, although they usually require a lot of work to get good results. But one can always dream of a "Stokowski" or "Karajan" neural network that interprets your MIDI composition with emotion and sensibility!
Re: What makes Bach sound like Bach? New dataset teaches algorithms classical music
#9i ran into a few issues trying to study classical music with a computer. first of all is merely putting in some representation of the musical score into a file. This was accomplished by MIDI but I am hoping for a more standard way that looks more like the notes of a score. Another problem is once you have the music there's a tremendous amount of "interpretation" that a musician does. the nodes may each read 1/8 but a…
You might look at GNU Lillypond, which might be the type of representation you are looking for, since it can be made to look like the notes of a score.
Here are some collections of music in that format:
Re: What makes Bach sound like Bach? New dataset teaches algorithms classical music
#10Relevant project to train a model that generates Bach music