Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spleeter: Extract voice, piano, drums, etc. from any music track

github.com

Post reply on HN
Print view View original

171–180 of 182 posts

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#171

Post by jononor »

rrss wrote: post #141
jononor wrote: post #125

Earlier quoted context omitted.

People are hard at work on denoising of audio using neural networks also. I would expect that if one trained denoisers for each type of source separated here, and passed the separated audio through them, things would get even better.

spleeter is using neural networks.

Yes, for source separation. Denoising is generally a separate task. A denoising network would take in a noisy signal for a single source and outputs a cleaned up one. It would be trained on the specific source type, for example vocals.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#172

Post by alez »

Tried it on “Halleluwah” by CAN, had to hear those drums:

https://soundcloud.com/alezzzz/can-halleluwah-drums-extracte...

Finding drum breaks in music is very time consuming. This is gonna be amazing for music production. Think how 90s jungle would’ve been if they had access to every drum take ever

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#173

Post by jmpavlec »

beachy wrote: post #39
lubujackson wrote: post #35

Earlier quoted context omitted.

Seems like most music (from the 70s on at least) is recorded multi-track and the data is out there, just not accessible to anybody. If you ever watch Rick Beato videos, he takes classic songs and isolates vocal/drum/etc. tracks all the time, I'm not sure how he has access to them: https://www.youtube.com/playlist?list=PLW0NGgv1qnfzb1klL6Vw9... But you probably don't need to bother with old recordings since there is S…

Rick says in one of his videos that he and some of his buddies have got old copies of the original source (separated) tracks, and they kind of pass them around between each other. I find that pretty amazing given the litigiousness of some in the music industry, but there we are. Side note: I discovered Rick Beato a few months ago and I've watched heaps of his videos. It's really fascinating hearing old classics torn…

Thanks for the Rick Beato mention. Just spent a couple hours watching some of his breakdowns. Fascinating stuff and reminded me how much I like this type of analysis.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#175

Post by ZeikJT »

seventhtiger wrote: post #163

Aren't all the most popular audio formats lossy? Extracting full data from lossy compression requires reconstruction. Even if they are able to completely extract all tracks they would have gaps and be very low quality.

If you want lossless audio in a decently popular format then just go looking for FLAC, you'll definitely find some. Even bandcamp uses it [1].

[1] - https://en.wikipedia.org/wiki/FLAC#Adoption_and_implementati...

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#176

Post by smrq »

alez wrote: post #172

Tried it on “Halleluwah” by CAN, had to hear those drums: https://soundcloud.com/alezzzz/can-halleluwah-drums-extracte... Finding drum breaks in music is very time consuming. This is gonna be amazing for music production. Think how 90s jungle would’ve been if they had access to every drum take ever

Wow, this is the isolated track I never knew I needed to hear.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#177

Post by exogen »

The extracted vocals sound great! But the resulting accompaniment tracks I've heard so far (tried on a handful of songs) aren't of usable quality for most purposes where you'd want an instrumental track – they're too sonically mangled.

Since people are often interested in doing this for a handful of specific tracks and not necessarily en masse, I'd be curious about what a human-assisted version of this could look like and whether you really could get near-perfect results...

What if you explicitly selected portions of the track you knew had vocals, so it could (1) know to leave the rest alone and (2) know what the backing track for the specific song naturally sounds like when there's no singing happening? It could try to match that sonic profile more carefully in the vocal-removed version.

Or what if you could give it even more info, and record yourself/another singing (isolated) over the track? Then it would have information about what phonemes it should expect to find and remove (and whatever effects like reverb are applied to them).

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#179

Post by pcf »

lubujackson wrote: post #35
voicedYoda wrote: post #7

I gave a talk at pycon this year about dsp [1], specifically some of the complexities surrounding this. I came across a few other ml projects that claimed to do this as well, and the biggest hold up is getting enough properly trained data, tagged appropriately, in order to let the models train correctly. in the git repo of this project they also explicitly state you need to train on your own data set, though you can…

Seems like most music (from the 70s on at least) is recorded multi-track and the data is out there, just not accessible to anybody. If you ever watch Rick Beato videos, he takes classic songs and isolates vocal/drum/etc. tracks all the time, I'm not sure how he has access to them: https://www.youtube.com/playlist?list=PLW0NGgv1qnfzb1klL6Vw9... But you probably don't need to bother with old recordings since there is S…

I've personally collected thousands of multitracks, stems and remix kits.

Sources are e.g. multitracks that someone leaked (like original unmixed Madonna sessions), constructed MOGG files from various Rock Band games, stems prepared for remixers etc.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#180

Post by aasasd »

kranner wrote: post #92
aasasd wrote: post #76

Earlier quoted context omitted.

I'd guess this would result in a model for separating SNES music.

I would guess Garageband tracks would be more representative of 'real' instruments than chiptunes.

BTW, in the play-along mode in GB where you get pre-recorded accompaniment tracks, you can replace the drummer's kit with a drum machine and hang some filters onto it. Much fun is to be had.

However, this reminds me that filters probably make things much harder for the separation model, with the explosion of possible sounds from an instrument or voice. (Vishudha Kali's music is a nice illustration of that.)

Post reply on HN