Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spleeter: Extract voice, piano, drums, etc. from any music track

github.com

Post reply on HN
Print view View original

131–140 of 182 posts

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#131

Post by IAmGraydon »

jacquesm wrote: post #112

This is very timely. I've been working for about 3 months now on a utility that transforms mp3's to midi files. It's a hard problem and even though I'm making steady progress the end is nowhere in sight. This will give me something to benchmark against with for instance voice accompanied by piano. Thank you for making/posting this. For an idea how this project is coming along: https://jacquesmattheij.com/toccata.mp3…

Just FYI in case you weren't aware - Ableton Live and several other DAWs have this capability built in. It's far from perfect, but great for humming a melody and then quickly turning it into MIDI.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#132

Post by jacquesm »

IAmGraydon wrote: post #131
jacquesm wrote: post #112

This is very timely. I've been working for about 3 months now on a utility that transforms mp3's to midi files. It's a hard problem and even though I'm making steady progress the end is nowhere in sight. This will give me something to benchmark against with for instance voice accompanied by piano. Thank you for making/posting this. For an idea how this project is coming along: https://jacquesmattheij.com/toccata.mp3…

Just FYI in case you weren't aware - Ableton Live and several other DAWs have this capability built in. It's far from perfect, but great for humming a melody and then quickly turning it into MIDI.

How good is it (% accuracy) for polyphony? I can upload the toccata original if you want.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#133

Post by CamperBob2 »

BrentOzar wrote: post #118
noja wrote: post #102

Earlier quoted context omitted.

So Jonathan Coulton is now the new Suzanne Vega?

> So Jonathan Coulton is now the new Suzanne Vega? Bravo. For people who didn't get the sublime reference, Suzanne Vega's song Tom's Diner was a benchmark test during development of the MP3.[1] [1]: https://en.wikipedia.org/wiki/Tom%27s_Diner#The_"Mother_of_t...

Tom's Diner wasn't just a benchmark. Brandenburg listened to it obsessively, over and over, to the exclusion of a lot of other content he might have done well to pay more attention to. Which is what tends to happen when you work on audio processing code, for better or worse.

So that's why the MP3 format mangles male vocals so badly at all but the highest bitrates. Now you know the Rest Of The Story... or at least you've read it on the Internet.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#134

Post by codedokode »

nickjj wrote: post #110
codedokode wrote: post #100

Earlier quoted context omitted.

While it's a great technology, the result sounds somewhat robotic. On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder, something is missing. The voice contains pieces of strumming sound. Guitar also sounds "blurred", as if someone cut an object from the picture and blurred the cut to make it less visible. Clap sound is distorted, on the o…

That roboticness is because there's overlap in frequencies between the voice and the instruments. I have no idea how this tool splits them up at the implementation level but I imagine it tries to split it up based on frequencies and when it lifts out the voice, it's cutting out a ton of frequencies that would normally be in your voice so now it sounds very unnatural, blocky and metallic. With studio quality headphone…

I don't argue that it's a great technology, but some of the neighbour commenters wrote things like "perfect", and to me it doesn't sound like "perfect" yet.

For example, in the beginning, if you listen to the phrase "from the office ...", in original recording it sounds smooth, like a single phrase (and the voice is warm and pleasant), but in the separated track it sounds like it is synthesized from pieces that are not properly connected, like vocaloid songs. It sounds little harsher. Transitions sound unnatural. And the phrase "heya Tom" in the separated track is split into "heya" and "Tom" with some unnatural sound between (or maybe in the beginning of "Tom"). It is like transitions you can hear in vocaloid tracks. Or the kind of artifacts you get if you over-compress an MP3 file.

And "it's good to see you" part also sounds robotic.

Maybe it's losing some of harmonics, but in a different way for different syllables and they don't sound like a single phrase anymore.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#135

Post by mikorym »

jacquesm wrote: post #115
codedokode wrote: post #100

Earlier quoted context omitted.

While it's a great technology, the result sounds somewhat robotic. On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder, something is missing. The voice contains pieces of strumming sound. Guitar also sounds "blurred", as if someone cut an object from the picture and blurred the cut to make it less visible. Clap sound is distorted, on the o…

> While it's a great technology, the result sounds somewhat robotic. That's like complaining about how bad the pig plays the violin. This is absolutely incredible. The complexity level for this problem is right off the scale and the software does a passable job of it. Given some time and more training data and a few more people working on it this has serious potential.

I think you meant how well the pig paints. [1] [2]

[1] https://pigcasso.org/wp-content/uploads/2018/12/7.jpg [2] Also, I am not affiliated with anything in particular that has been mentioned, or with the pig.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#136

Post by jacquesm »

mikorym wrote: post #135
jacquesm wrote: post #115

Earlier quoted context omitted.

> While it's a great technology, the result sounds somewhat robotic. That's like complaining about how bad the pig plays the violin. This is absolutely incredible. The complexity level for this problem is right off the scale and the software does a passable job of it. Given some time and more training data and a few more people working on it this has serious potential.

I think you meant how well the pig paints. [1] [2] [1] https://pigcasso.org/wp-content/uploads/2018/12/7.jpg [2] Also, I am not affiliated with anything in particular that has been mentioned, or with the pig.

I think the original was in relation to pigs dancing.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#138

Post by viburnum »

IAmGraydon wrote: post #131
jacquesm wrote: post #112

This is very timely. I've been working for about 3 months now on a utility that transforms mp3's to midi files. It's a hard problem and even though I'm making steady progress the end is nowhere in sight. This will give me something to benchmark against with for instance voice accompanied by piano. Thank you for making/posting this. For an idea how this project is coming along: https://jacquesmattheij.com/toccata.mp3…

Just FYI in case you weren't aware - Ableton Live and several other DAWs have this capability built in. It's far from perfect, but great for humming a melody and then quickly turning it into MIDI.

How do you do this in Ableton?

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#139

Post by alez »

IAmGraydon wrote: post #131
jacquesm wrote: post #112

This is very timely. I've been working for about 3 months now on a utility that transforms mp3's to midi files. It's a hard problem and even though I'm making steady progress the end is nowhere in sight. This will give me something to benchmark against with for instance voice accompanied by piano. Thank you for making/posting this. For an idea how this project is coming along: https://jacquesmattheij.com/toccata.mp3…

Just FYI in case you weren't aware - Ableton Live and several other DAWs have this capability built in. It's far from perfect, but great for humming a melody and then quickly turning it into MIDI.

There’s a pretty cool library from Googles Magenta team that does piano transcription pretty well. https://magenta.tensorflow.org/onsets-frames

They say it’s only really good for piano, but I definitely use it for all kinds of samples. Great for inspiration

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#140

Post by danso »

antman wrote: post #71

How could we extract anything but the voice e.g. karaoke?

The repo's quick start instructions [0] show how to use it with the "2-stems" model [1], which separate the source audio into two files: output/source/vocals.wav and outputdir/source/accompaniment.wav:

    $ spleeter separate -i spleeter/source.mp3 \
         -p spleeter:2stems \
         -o outputdir
[0] https://github.com/deezer/spleeter#quick-start

[1] https://github.com/deezer/spleeter/wiki/2.-Getting-started#u...

Post reply on HN