21–29 of 29 posts
This is going to be huge for the hard of hearing.
Unlike reading, I don't think audio can convey the same meaning in a different sensory format. At best, they perceive it but in an alien way to most people. It's like describing a painting in musical notes.
I wonder if this can segregate vocals from instrumentals in a mix? That would be great for mashups.
Incidentally, due to the way a lot of stereo tracks are mixed, it's often possible to mostly remove the vocal track from a song. I'm more curious if this algorithm could perform the reverse task - playing the vocals only. My intuition is that the results would be poor because of the wide human vocal range and the fact that words need to be discernible, not just notes. But I would love to be proven wrong here.
Interesting. I wonder how well a logistic regression that spits out masks would perform in the source separation task. Also a bit surprising to see that they had to STFT the audio before feeding it into a convnet. I thought half the point of convnets was that they figure out how to do spectral domain representations on their own...
Earlier quoted context omitted.
Incidentally, due to the way a lot of stereo tracks are mixed, it's often possible to mostly remove the vocal track from a song. I'm more curious if this algorithm could perform the reverse task - playing the vocals only. My intuition is that the results would be poor because of the wide human vocal range and the fact that words need to be discernible, not just notes. But I would love to be proven wrong here.
If you can remove the vocals from a piece, you can then subtract that from the original to get just the vocals.
However, the right - left mix isn't exactly the song minus the vocals, it's an odd off-version, so subtracting that from the original song will leave mostly the vocals but with artifacts from the difference between the song truly without vocals and the right - left mix's interpretation thereof
Earlier quoted context omitted.
Incidentally, due to the way a lot of stereo tracks are mixed, it's often possible to mostly remove the vocal track from a song. I'm more curious if this algorithm could perform the reverse task - playing the vocals only. My intuition is that the results would be poor because of the wide human vocal range and the fact that words need to be discernible, not just notes. But I would love to be proven wrong here.
If you can remove the vocals from a piece, you can then subtract that from the original to get just the vocals.
Earlier quoted context omitted.
If you can remove the vocals from a piece, you can then subtract that from the original to get just the vocals.
Well that's not quite true. The point is, I believe, that vocals are generally put right in the center of the sound-stage, so they play equally in the left and right channels. Thus right - left is most of the rest of the song, but the vocals cancelled each other out. However, the right - left mix isn't exactly the song minus the vocals, it's an odd off-version, so subtracting that from the original song will leave mo…
Interesting. I wonder how well a logistic regression that spits out masks would perform in the source separation task. Also a bit surprising to see that they had to STFT the audio before feeding it into a convnet. I thought half the point of convnets was that they figure out how to do spectral domain representations on their own...
in theory yes, but in practice, giving the network the full information in the right format is crucial to have it train well and quickly.
Earlier quoted context omitted.
Well that's not quite true. The point is, I believe, that vocals are generally put right in the center of the sound-stage, so they play equally in the left and right channels. Thus right - left is most of the rest of the song, but the vocals cancelled each other out. However, the right - left mix isn't exactly the song minus the vocals, it's an odd off-version, so subtracting that from the original song will leave mo…
Yes, certainly. My point is that if you have "mostly no vocals" you can subtract that from the left + right mix to get "mostly the vocals". It won't be exactly right, sure.