Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spleeter: Extract voice, piano, drums, etc. from any music track

github.com

Post reply on HN
Print view View original

101–110 of 182 posts

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#101

Post by MattRix »

codedokode wrote: post #100
mwcampbell wrote: post #49

For your listening pleasure, here's a full-length demo. I decided to use the Jonathan Coulton classic "Re Your Brains", because I can legally share and modify his music under its Creative Commons license. First, the original: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Now the derived stems: Vocals: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Accompaniment: https://mwcampbell.u…

While it's a great technology, the result sounds somewhat robotic. On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder, something is missing. The voice contains pieces of strumming sound. Guitar also sounds "blurred", as if someone cut an object from the picture and blurred the cut to make it less visible. Clap sound is distorted, on the o…

It's easy to pick up on lots of little things, but this is still extremely impressive. It's also more than good enough for people who want to practice singing/playing over the song by themselves.

I think the next step would be to train a network that can un-robotify songs and then run it on this.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#102

Post by noja »

mwcampbell wrote: post #49

For your listening pleasure, here's a full-length demo. I decided to use the Jonathan Coulton classic "Re Your Brains", because I can legally share and modify his music under its Creative Commons license. First, the original: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Now the derived stems: Vocals: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Accompaniment: https://mwcampbell.u…

So Jonathan Coulton is now the new Suzanne Vega?

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#103

Post by noja »

BeeBoBub wrote: post #38

I am working on a product which makes use of this technology. I generate vocal pitch visualizations for karaoke http://pitchperfected.io

Cool - but your website needs some work. It looks like a landing page to gather interest rather than something backed by a real product. Show us some videos and singing, before and after, etc.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#106

Post by uryga »

codedokode wrote: post #100
mwcampbell wrote: post #49

For your listening pleasure, here's a full-length demo. I decided to use the Jonathan Coulton classic "Re Your Brains", because I can legally share and modify his music under its Creative Commons license. First, the original: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Now the derived stems: Vocals: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Accompaniment: https://mwcampbell.u…

While it's a great technology, the result sounds somewhat robotic. On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder, something is missing. The voice contains pieces of strumming sound. Guitar also sounds "blurred", as if someone cut an object from the picture and blurred the cut to make it less visible. Clap sound is distorted, on the o…

> On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder

funny, the original recording seemed kind of robotic to me! maybe not robotic, but like it's been filtered somehow. but that might just be my not-so-great headphones

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#107

Post by wpietri »

beachy wrote: post #39
lubujackson wrote: post #35

Earlier quoted context omitted.

Seems like most music (from the 70s on at least) is recorded multi-track and the data is out there, just not accessible to anybody. If you ever watch Rick Beato videos, he takes classic songs and isolates vocal/drum/etc. tracks all the time, I'm not sure how he has access to them: https://www.youtube.com/playlist?list=PLW0NGgv1qnfzb1klL6Vw9... But you probably don't need to bother with old recordings since there is S…

Rick says in one of his videos that he and some of his buddies have got old copies of the original source (separated) tracks, and they kind of pass them around between each other. I find that pretty amazing given the litigiousness of some in the music industry, but there we are. Side note: I discovered Rick Beato a few months ago and I've watched heaps of his videos. It's really fascinating hearing old classics torn…

Isn't that litigiousness mainly around money? Another important currency in the music world is respect. E.g., look at how Chamillionaire talks about Weird Al: http://yankovic.org/blog/2006/09/13/high-praise-from-chamill...

Given that, I expect that a show titled "What Makes This Song Great" will do fine. Who doesn't love having somebody note the non-obviously good parts of their work. Especially if, as with Weird Al, proper royalties are paid.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#108

Post by voicedYoda »

_fbpt wrote: post #55
voicedYoda wrote: post #7

I gave a talk at pycon this year about dsp [1], specifically some of the complexities surrounding this. I came across a few other ml projects that claimed to do this as well, and the biggest hold up is getting enough properly trained data, tagged appropriately, in order to let the models train correctly. in the git repo of this project they also explicitly state you need to train on your own data set, though you can…

The SNES is a 1990s game console. Its music is generally synthesized by the SPC700 chip, from individual instruments stored in 64 kilobytes of RAM (so the instruments often sound synthetic and muffled). The advantage is that it's possible to separate out instruments. Either: - Programatically gather a list of all samples used in the song - Generate many modified .spc files, each of which mutes 1 sample via editing th…

I did come across the person who did a similar project (automating instruments based on previously recorded music), however in one project that was playing live instruments from an NES, the signals were already separated. That said, I'm not following the context of your response to my post.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#109

Post by cantankerous »

orloffm wrote: post #105

Played with it. The quality of the result is mostly dependent on the amount of clipping in the source file. Basically, all post-90s masters produce weird results with orcs singing in the background. And classics from 60s yield fantastic results.

Might be the training data?

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#110

Post by nickjj »

codedokode wrote: post #100
mwcampbell wrote: post #49

For your listening pleasure, here's a full-length demo. I decided to use the Jonathan Coulton classic "Re Your Brains", because I can legally share and modify his music under its Creative Commons license. First, the original: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Now the derived stems: Vocals: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Accompaniment: https://mwcampbell.u…

While it's a great technology, the result sounds somewhat robotic. On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder, something is missing. The voice contains pieces of strumming sound. Guitar also sounds "blurred", as if someone cut an object from the picture and blurred the cut to make it less visible. Clap sound is distorted, on the o…

That roboticness is because there's overlap in frequencies between the voice and the instruments.

I have no idea how this tool splits them up at the implementation level but I imagine it tries to split it up based on frequencies and when it lifts out the voice, it's cutting out a ton of frequencies that would normally be in your voice so now it sounds very unnatural, blocky and metallic.

With studio quality headphones I can notice a massive difference for the worse between the original and separated vocal track. It reminds me of when I turned on a noise gate too high when recording audio for my courses. That noise gate clamps down on certain frequencies to help eliminate room noise but it also dampens or removes natural frequencies that occur on the low end of an average male voice. It gives that same very jagged sounding audio waveform.

Post reply on HN