Spleeter: Extract voice, piano, drums, etc. from any music track • Hacker News Board

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#41

Post by lrobinovitch » Sun, Nov 03, 2019, 4:26 AM UTC

ooobo wrote: post #15

Gave this a go, it's an easy install with pip, and results are pretty quick even on an old macbook. Splits into 2stems (vocals/accompaniment) on some random songs I chose actually quite good using the pretrained models provided. Of course, ripping the vocals out of the accompaniment takes out a good chunk of the middle frequencies so some songs sound a bit wonky. Worth a play if you are interested.

Same, Rage Against The Machine - Killing In the Name came out sounding great. Very cool.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#42

Post by murat124 » Sun, Nov 03, 2019, 4:30 AM UTC

I gave it a try with Megadeth's Holy wars[1], was expecting something like this[2] but got very deep audio. Not sure why but perhaps it's because bassist David Ellefson uses pick which gives the percussive sound and it suits to Megadeth.

Any parameter I could use with spleeter to get a similar output?

[1] https://www.youtube.com/watch?v=9d4ui9q7eDM

[2] https://www.youtube.com/watch?v=uWkykQHsJ-Y

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#43

Post by sillysaurusx » Sun, Nov 03, 2019, 4:31 AM UTC

tomo-makes wrote: post #20

dgreensp wrote: post #5

Are there any examples I can listen to?

I set up a Colab notebook to try spleeter out for myself. You can try picking up your favorite mp3, renaming it to "audio_sample.mp3", uploading it to the Colab, and spining all the cells on the notebook. Enjoy. https://colab.research.google.com/gist/tomo-makes/33b9bc7f22...

This is great. Thank you! I extended the notebook with an example of downloading a youtube video, extracting the audio, then feeding it through spleeter.

Notebook: https://colab.research.google.com/gist/shawwn/0f286f5d4bc22e...

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#44

Post by matchagaucho » Sun, Nov 03, 2019, 4:38 AM UTC

Intermernet wrote: post #37

I'd love to see how this compares with Celemony Melodyne. As far as I've been able to determine, Melodyne doesn't use ML, but it's hard to find out exactly what it does use. Either way, an open source competitor to Melodyne is a welcome addition!

I’ve always assumed Melodyne uses FFT bins.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#45

Post by ErikAugust » Sun, Nov 03, 2019, 4:41 AM UTC

theLotusGambit wrote: post #28

Not only are the results good, but the music is generated decently rapidly. The implications are clear: whoever wants to make a quick fortune on YouTube should start converting and uploading truckloads of songs as fast as possible. The demand is there. I could easily see that bringing in millions of views.

They’d still get tagged for copyright.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#46

Post by sehugg » Sun, Nov 03, 2019, 4:55 AM UTC

ooobo wrote: post #15

Gave this a go, it's an easy install with pip, and results are pretty quick even on an old macbook. Splits into 2stems (vocals/accompaniment) on some random songs I chose actually quite good using the pretrained models provided. Of course, ripping the vocals out of the accompaniment takes out a good chunk of the middle frequencies so some songs sound a bit wonky. Worth a play if you are interested.

It's really good on the 2-stem stuff. On the 4-stem model, it's a bit shy about the bass part, and parts drift in and out. I'd like to try it on a FLAC.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#47

Post by megablast » Sun, Nov 03, 2019, 5:07 AM UTC

xamuel wrote: post #29

esfandia wrote: post #27

Karaoke with the most obscure songs!

And even better, karaoke that doesn't suck--most karaoke tracks are covers by cheap bands and you can clearly tell the inferior quality if you're familiar with the song.

Sure, not to mention the awful singing accompanying most karaoke tracks.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#48

Post by jMyles » Sun, Nov 03, 2019, 5:09 AM UTC

Somewhat of a tangent, but does anyone have a recommendation of an open source (ideally python) program that can make MIDI from piano audio?

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#49

Post by mwcampbell » Sun, Nov 03, 2019, 5:14 AM UTC

For your listening pleasure, here's a full-length demo. I decided to use the Jonathan Coulton classic "Re Your Brains", because I can legally share and modify his music under its Creative Commons license.

First, the original:

https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-...

Now the derived stems:

Vocals: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-...

Accompaniment: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-...

Note: I'm not affiliated with this project or Mr. Coulton. I just think this is a cool project and wanted to share.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#50

Post by lreichold » Sun, Nov 03, 2019, 5:34 AM UTC

An interesting alternative approach for instrument sound separation is to use a fused audio + video model. So, given that you also have video of the instruments being played, you can perform this separation with higher fidelity.

I was fascinated by the work done by “The Sound of Pixels” project at MIT.

http://sound-of-pixels.csail.mit.edu/