Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spleeter: Extract voice, piano, drums, etc. from any music track

github.com

Post reply on HN
Print view View original

161–170 of 182 posts

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#161

Post by mwcampbell »

rafa1981 wrote: post #144
codedokode wrote: post #100

Earlier quoted context omitted.

While it's a great technology, the result sounds somewhat robotic. On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder, something is missing. The voice contains pieces of strumming sound. Guitar also sounds "blurred", as if someone cut an object from the picture and blurred the cut to make it less visible. Clap sound is distorted, on the o…

Maybe the source was compressed audio instead of flac/wav? Edit: the source is an mp3, which removes audio frequencies based on perception/masking with other frequencies. It's perfectly normal that it is showing artifacts. A better source is needed.

I encoded the source in MP3 for your convenience. But the source that I fed into Spleeter was FLAC.

Edit: I uploaded the original FLAC to my web server last night, before I decided that MP3 would be more convenient:

https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-...

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#162

Post by mwcampbell »

savrajsingh wrote: post #149
mwcampbell wrote: post #49

For your listening pleasure, here's a full-length demo. I decided to use the Jonathan Coulton classic "Re Your Brains", because I can legally share and modify his music under its Creative Commons license. First, the original: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Now the derived stems: Vocals: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Accompaniment: https://mwcampbell.u…

Did it just work or did you have to supply something beyond the original JC track?

Just worked.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#164

Post by mwcampbell »

uryga wrote: post #106
codedokode wrote: post #100

Earlier quoted context omitted.

While it's a great technology, the result sounds somewhat robotic. On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder, something is missing. The voice contains pieces of strumming sound. Guitar also sounds "blurred", as if someone cut an object from the picture and blurred the cut to make it less visible. Clap sound is distorted, on the o…

> On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder funny, the original recording seemed kind of robotic to me! maybe not robotic, but like it's been filtered somehow. but that might just be my not-so-great headphones

The original lead vocal is definitely processed. You can hear that processing clearly if you apply the classic vocal removal (really, center removal) effect. (My favorite implementation of that is the Center Cut DSP for foobar2000.)

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#165

Post by quadrangle »

vonseel wrote: post #87
jMyles wrote: post #48

Somewhat of a tangent, but does anyone have a recommendation of an open source (ideally python) program that can make MIDI from piano audio?

Melodyne or Ableton. I've found Melodyne to be more accurate, but still not perfect.

They asked for open source

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#166

Post by jMyles »

jacquesm wrote: post #117
jMyles wrote: post #48

Somewhat of a tangent, but does anyone have a recommendation of an open source (ideally python) program that can make MIDI from piano audio?

I've been working on this for the last 3 months. Can you send me your audio file? I'll send you back the midi I can generate; it won't be perfect but it might be usable. See comment elsewhere in this thread.

Awesome! Is your project open source? Can't wait to see it.

Here's the first (but by no means only) file I have in mind:

http://ocrmirror.org/files/music/albums/ff7/MP3/3-02%20Shnab...

Stone Eyes from the Final Fantasy VII tribute album Voices of the Lifestream.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#168

Post by tianshuo »

jMyles wrote: post #48

Somewhat of a tangent, but does anyone have a recommendation of an open source (ideally python) program that can make MIDI from piano audio?

Here's an open-source project from Google Deepmind's Magenta Project, that does exactly what you want. https://magenta.tensorflow.org/onsets-frames

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#169

Post by roryokane »

czr wrote: post #56

messed around with the 2stem model for a bit and it's reasonably good. I think phonicmind is still a bit better - phonicmind tends to err on the side of keeping too much, while the 2stem model tries to isolate aggressively and often damages the vocal as a result (distorting words by losing some harmonics, or losing quiet words entirely) example: https://files.catbox.moe/wjruiv.mp3 (phonicmind) https://files.catbox.mo…

For those who, like me, hadn’t heard of PhonicMind before, it’s an online service at https://phonicmind.com/ that charges $4 to $1.5 per song to separate out vocals, drums, bass, and the rest of the sounds. You can upload any audio file to that website and get a 30-second preview of separated parts for it.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#170

Post by navjack27 »

marcan_42 wrote: post #88
aasasd wrote: post #74

Earlier quoted context omitted.

I'd guess extract the voice and then subtract it from the rest with something like Audacity. I'm not sure which operation would do that, but I believe that it exists. Also, other comments here speak of “separating into the voice and accompaniment,” so maybe the model/program already do exactly what you need.

Invert and mix. When you have an instrumental version of a song (from the same stems as the vocal version) this is already one way to get the vocals out without any fancy machine learning. The main tricks besides what you can do in Audacity like that are properly time-aligning the tracks (even if they drift a bit) and compensating for phase issues and compression. I wrote a dirty tool that does that and I've been mea…

I've been doing something like this for a bit in Audition. Center channel extract > invert phase > save as wav > create multi-track project > add original > add modified > up the volume on the vocal extracted modified version until the vocals go away

Or you can do the exact opposite and instead of center channel extracting the vocals you can remove the vocals and use this method to better isolate vocals.

Although if things do fun stuff with stereo it might not be exact.

Post reply on HN