Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spleeter: Extract voice, piano, drums, etc. from any music track

github.com

Post reply on HN
Print view View original

141–150 of 182 posts

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#141

Post by rrss »

jononor wrote: post #125
codedokode wrote: post #100

Earlier quoted context omitted.

While it's a great technology, the result sounds somewhat robotic. On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder, something is missing. The voice contains pieces of strumming sound. Guitar also sounds "blurred", as if someone cut an object from the picture and blurred the cut to make it less visible. Clap sound is distorted, on the o…

People are hard at work on denoising of audio using neural networks also. I would expect that if one trained denoisers for each type of source separated here, and passed the separated audio through them, things would get even better.

spleeter is using neural networks.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#142

Post by jacquesm »

alez wrote: post #139
IAmGraydon wrote: post #131

Earlier quoted context omitted.

Just FYI in case you weren't aware - Ableton Live and several other DAWs have this capability built in. It's far from perfect, but great for humming a melody and then quickly turning it into MIDI.

There’s a pretty cool library from Googles Magenta team that does piano transcription pretty well. https://magenta.tensorflow.org/onsets-frames They say it’s only really good for piano, but I definitely use it for all kinds of samples. Great for inspiration

Oh cool, thank you for that, I did not know about this yet. That may come in very handy.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#143

Post by amelius »

neohaven wrote: post #93
amelius wrote: post #91

> Spleeter is the Deezer source separation library with pretrained models Curious, what is Deezer using this for?

Gonna guess beat/mood/song analysis.

Yeah but they can use the raw song for that I suppose.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#144

Post by rafa1981 »

codedokode wrote: post #100
mwcampbell wrote: post #49

For your listening pleasure, here's a full-length demo. I decided to use the Jonathan Coulton classic "Re Your Brains", because I can legally share and modify his music under its Creative Commons license. First, the original: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Now the derived stems: Vocals: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Accompaniment: https://mwcampbell.u…

While it's a great technology, the result sounds somewhat robotic. On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder, something is missing. The voice contains pieces of strumming sound. Guitar also sounds "blurred", as if someone cut an object from the picture and blurred the cut to make it less visible. Clap sound is distorted, on the o…

Maybe the source was compressed audio instead of flac/wav?

Edit: the source is an mp3, which removes audio frequencies based on perception/masking with other frequencies. It's perfectly normal that it is showing artifacts. A better source is needed.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#145

Post by jacquesm »

alez wrote: post #139
IAmGraydon wrote: post #131

Earlier quoted context omitted.

Just FYI in case you weren't aware - Ableton Live and several other DAWs have this capability built in. It's far from perfect, but great for humming a melody and then quickly turning it into MIDI.

There’s a pretty cool library from Googles Magenta team that does piano transcription pretty well. https://magenta.tensorflow.org/onsets-frames They say it’s only really good for piano, but I definitely use it for all kinds of samples. Great for inspiration

So, I used it to run the same toccata test, here are the results:

The Magenta code:

f_measure 71.56 precision 65.75 recall 78.49 accuracy 55.72

My little batch of code:

f_measure 77.74 precision 93.40 recall 66.57 accuracy 63.58

Interpreting the results is tricky, they are obviously better on 'recall' but that is at the expense of being much less precise which gives a much better result for my code; besides it is nicer to listen to because there are far fewer spurious notes.

My code also runs about 100 times as fast and uses very little in terms of resources. So, rather than being depressed it looks like I'm on to something :)

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#148

Post by smcnally »

viburnum wrote: post #138
IAmGraydon wrote: post #131

Earlier quoted context omitted.

Just FYI in case you weren't aware - Ableton Live and several other DAWs have this capability built in. It's far from perfect, but great for humming a melody and then quickly turning it into MIDI.

How do you do this in Ableton?

https://www.ableton.com/en/manual/converting-audio-to-midi/

There are four options.

> Convert Melody to New MIDI Track

> This command identifies the pitches in monophonic audio and places them into a clip on a new MIDI track.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#149

Post by savrajsingh »

mwcampbell wrote: post #49

For your listening pleasure, here's a full-length demo. I decided to use the Jonathan Coulton classic "Re Your Brains", because I can legally share and modify his music under its Creative Commons license. First, the original: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Now the derived stems: Vocals: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Accompaniment: https://mwcampbell.u…

Did it just work or did you have to supply something beyond the original JC track?

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#150

Post by alez »

jacquesm wrote: post #145
alez wrote: post #139

Earlier quoted context omitted.

There’s a pretty cool library from Googles Magenta team that does piano transcription pretty well. https://magenta.tensorflow.org/onsets-frames They say it’s only really good for piano, but I definitely use it for all kinds of samples. Great for inspiration

So, I used it to run the same toccata test, here are the results: The Magenta code: f_measure 71.56 precision 65.75 recall 78.49 accuracy 55.72 My little batch of code: f_measure 77.74 precision 93.40 recall 66.57 accuracy 63.58 Interpreting the results is tricky, they are obviously better on 'recall' but that is at the expense of being much less precise which gives a much better result for my code; besides it is nic…

Oh wow that's extremely promising! Yeah the magenta thing destroys my browser when I run it. Still feels like magic though haha. I would be extremely interested in some other options so good luck!
Post reply on HN