A reflection about automatic transcription of music (2018)
seventhstring.com
A reflection about automatic transcription of music (2018)
1–10 of 24 posts
Re: A reflection about automatic transcription of music (2018)
#2It seems like an interesting machine learning problem rather than AI complete? There are lots of finicky details when transcribing speech as well, but it's apparently not AI complete. You do need a large corpus so the algorithm knows what's typical.
If I were going to work on this, I would work on generating lead sheets from YouTube videos. Recognizing chords seems like a useful thing to solve?
Re: A reflection about automatic transcription of music (2018)
#3All true, although one of the things that bothers me is that even if you can read music you to need to understand the historical context in order to play a piece properly. For example, you would play a Bach piece with a lighter touch and less legato than you would a Beethoven piece with the same notes. And of course a jazz musician looking at music from a certain area is almost certainly going to play what look like eighth notes with swing, which is a completely different feel and should actually be notated differently as well.
Re: A reflection about automatic transcription of music (2018)
#4It seems like an interesting machine learning problem rather than AI complete? There are lots of finicky details when transcribing speech as well, but it's apparently not AI complete. You do need a large corpus so the algorithm knows what's typical. If I were going to work on this, I would work on generating lead sheets from YouTube videos. Recognizing chords seems like a useful thing to solve?
It’s so so much harder than transcription of language. The article attempts to capture this point, but while there’s generally only a few ways to punctuate a sentence, assuming you have all the words right, there are thousands of potential ways to notate a musical phrase, even assuming you can correctly interpret all the notes and rhythms correctly, but only a few would be easily sensible to a human performer.
Re: A reflection about automatic transcription of music (2018)
#5All true, although one of the things that bothers me is that even if you can read music you to need to understand the historical context in order to play a piece properly. For example, you would play a Bach piece with a lighter touch and less legato than you would a Beethoven piece with the same notes. And of course a jazz musician looking at music from a certain area is almost certainly going to play what look like…
The notation is shorthand for the composers' intent, not a computer program to perfectly reconstruct the music every time. Ambiguity is a feature, not a bug.
Re: A reflection about automatic transcription of music (2018)
#6All true, although one of the things that bothers me is that even if you can read music you to need to understand the historical context in order to play a piece properly. For example, you would play a Bach piece with a lighter touch and less legato than you would a Beethoven piece with the same notes. And of course a jazz musician looking at music from a certain area is almost certainly going to play what look like…
Not sure why this bothers you. Musical notation is, by necessity, a greatly simplified representation of the intended performance. Not only that, but the composers themselves well understand that the interpretation of their notation is itself an artistic activity that is independent from the musical notation they write. There are thousands of recorded performances of any particular Bach or Beethoven work, every one different, sometimes very very much different. And none of them are “wrong”. We certainly don’t know for sure how Bach’s and Beethoven’s works were performed in their own days—there’s uncertainty over tuning, dynamics, and tempo, to say nothing of stylistic details—but we do know that the instruments they had available were very different, and yet their works are still great today, even though we can never reproduce what they were “supposed” to sound like.
Re: A reflection about automatic transcription of music (2018)
#7It seems like an interesting machine learning problem rather than AI complete? There are lots of finicky details when transcribing speech as well, but it's apparently not AI complete. You do need a large corpus so the algorithm knows what's typical. If I were going to work on this, I would work on generating lead sheets from YouTube videos. Recognizing chords seems like a useful thing to solve?
>Recognizing chords seems like a useful thing to solve?
It's also pretty easy to teach and learn for a practicing musician but much more difficult to teach a machine, since you run into issues with blind source separation.
Speech transcription is good enough provided we have enough preprocessing power, assume a single speaker, and know the language beforehand and have trained the model on a large number of previous speakers with the same dialect/accent.
In music, you don't know how many speakers there are (instruments playing), the dialect/accent (orchestration/chord voicing) changes on the fly, representations are non-unique and contextual, and artists intentionally subvert expected results to make good music.
Humans are just better at this and easier to train to do it than computers, for the moment.
Re: A reflection about automatic transcription of music (2018)
#8All true, although one of the things that bothers me is that even if you can read music you to need to understand the historical context in order to play a piece properly. For example, you would play a Bach piece with a lighter touch and less legato than you would a Beethoven piece with the same notes. And of course a jazz musician looking at music from a certain area is almost certainly going to play what look like…
"Swing" was known even in the Baroque era, actually. It was called notes inégales (which literally means "non-equal notes"), while playing eighths "straight" would be called notes égales ("equal notes"). Some of these other variations are inherent in how notation has evolved over time - by the time of Beethoven, lots of indications would be written into the piece to try and convey how it should be played. You don't get that kind of thing from Bach-era scores, and sometimes people play them a bit too mechanically as a result. It's very much a matter of finding the right balance for every subgenre or tradition.
Re: A reflection about automatic transcription of music (2018)
#9It seems like an interesting machine learning problem rather than AI complete? There are lots of finicky details when transcribing speech as well, but it's apparently not AI complete. You do need a large corpus so the algorithm knows what's typical. If I were going to work on this, I would work on generating lead sheets from YouTube videos. Recognizing chords seems like a useful thing to solve?
> There are lots of finicky details when transcribing speech as well, but it's apparently not AI complete. You do need a large corpus so the algorithm knows what's typical.
Speech is dramatically easier. The reason is that language is meant to communicate and consequently carries a lot of redundant information that you can bring to bear.
Music often has no such redundancy. It may have themes that differ slightly each time, so nothing to lock onto.
> Recognizing chords seems like a useful thing to solve
Except that a single C note (especially on stringed instruments) may have many harmonics that also look like a C chord. The problem isn't straightforward.
Re: A reflection about automatic transcription of music (2018)
#10Transcribing solo piano to MIDI, even very complex polyphonic pieces, is fairly reasonable these days. You can try a demo from a not quite state of the art system here: https://piano-scribe.glitch.me/ . (full disclosure, I was one of researchers who produced this system).
I would argue that there's a lot more people that can take the output of the MIDI and turn it into a reasonable score than there are people that can listen to the raw audio and turn it into a reasonable approximation of the notes.
Papers / dataset:
https://arxiv.org/pdf/1810.12247.pdf [New dataset and slightly improved network]
https://arxiv.org/pdf/1710.11153.pdf [Original Network]