I am working on a product which makes use of this technology. I generate vocal pitch visualizations for karaoke http://pitchperfected.io
FYI your email confirmation is going straight to spam on gmail. I'd recommend reaching out to Mailchimp.
Spleeter: Extract voice, piano, drums, etc. from any music track
81–90 of 182 posts
Re: Spleeter: Extract voice, piano, drums, etc. from any music track
#82I wonder how this compares to Open Unmix (https://github.com/sigsep/open-unmix-pytorch), that one calls itself state-of-the-art as well and is done in collaboration with Sony from what I see of their paper.
Re: Spleeter: Extract voice, piano, drums, etc. from any music track
#83Earlier quoted context omitted.
I set up a Colab notebook to try spleeter out for myself. You can try picking up your favorite mp3, renaming it to "audio_sample.mp3", uploading it to the Colab, and spining all the cells on the notebook. Enjoy. https://colab.research.google.com/gist/tomo-makes/33b9bc7f22...
This is great. Thank you! I extended the notebook with an example of downloading a youtube video, extracting the audio, then feeding it through spleeter. Notebook: https://colab.research.google.com/gist/shawwn/0f286f5d4bc22e...
Cool! That's so handy.
Re: Spleeter: Extract voice, piano, drums, etc. from any music track
#84I wonder how this compares to Open Unmix ( https://github.com/sigsep/open-unmix-pytorch ), that one calls itself state-of-the-art as well and is done in collaboration with Sony from what I see of their paper.
Oh I just found out their paper, http://archives.ismir.net/ismir2019/latebreaking/000036.pdf. It's pretty competitive.
Re: Spleeter: Extract voice, piano, drums, etc. from any music track
#85For your listening pleasure, here's a full-length demo. I decided to use the Jonathan Coulton classic "Re Your Brains", because I can legally share and modify his music under its Creative Commons license. First, the original: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Now the derived stems: Vocals: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Accompaniment: https://mwcampbell.u…
Holly cow! The separation is sort of perfect! Thanks for the demo.
Re: Spleeter: Extract voice, piano, drums, etc. from any music track
#86Earlier quoted context omitted.
That's the second time I've seen someone mention Melodyne for separating vocals from a full song source - I don't think that's something it can do? Melodyne is for tuning vocals / instruments & correcting timing on already isolated tracks.
melodyne's editing interface lets you remove different notes from a polyphonic track. so if it's just vocals + other tonal sounds, you can manually remove the other tonal sounds. example: https://youtu.be/2ZjdDatxTaQ?t=83
Hmm, never tried that with melodyne myself and the video you posted isn't a great example of an accurate vocal extraction - those are more like vocal chops and are already pretty dirty to begin with. Based on my experience with Melodyne, I'd be surprised if you could cleanly extract a plain singing vocal without tons and tons of work.
Re: Spleeter: Extract voice, piano, drums, etc. from any music track
#87Somewhat of a tangent, but does anyone have a recommendation of an open source (ideally python) program that can make MIDI from piano audio?
Melodyne or Ableton. I've found Melodyne to be more accurate, but still not perfect.
Re: Spleeter: Extract voice, piano, drums, etc. from any music track
#88How could we extract anything but the voice e.g. karaoke?
I'd guess extract the voice and then subtract it from the rest with something like Audacity. I'm not sure which operation would do that, but I believe that it exists. Also, other comments here speak of “separating into the voice and accompaniment,” so maybe the model/program already do exactly what you need.
Invert and mix.
When you have an instrumental version of a song (from the same stems as the vocal version) this is already one way to get the vocals out without any fancy machine learning. The main tricks besides what you can do in Audacity like that are properly time-aligning the tracks (even if they drift a bit) and compensating for phase issues and compression. I wrote a dirty tool that does that and I've been meaning to turn it into some kind of nicer GUI version.
Re: Spleeter: Extract voice, piano, drums, etc. from any music track
#89I gave a talk at pycon this year about dsp [1], specifically some of the complexities surrounding this. I came across a few other ml projects that claimed to do this as well, and the biggest hold up is getting enough properly trained data, tagged appropriately, in order to let the models train correctly. in the git repo of this project they also explicitly state you need to train on your own data set, though you can…
Seems like most music (from the 70s on at least) is recorded multi-track and the data is out there, just not accessible to anybody. If you ever watch Rick Beato videos, he takes classic songs and isolates vocal/drum/etc. tracks all the time, I'm not sure how he has access to them: https://www.youtube.com/playlist?list=PLW0NGgv1qnfzb1klL6Vw9... But you probably don't need to bother with old recordings since there is S…
I'm wondering if you could even use this to separate unrelated pieces of audio? E.g. instrumental music and someone reading a book out loud. And if you could use this to generate useful training data.
Re: Spleeter: Extract voice, piano, drums, etc. from any music track
#90Can this also separate the backing vocals from the lead singer?