Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Spleeter: Extract voice, piano, drums, etc. from any music track

github.com

Post reply on HN
Print view View original

111–120 of 182 posts

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#111

Post by voicedYoda »

marcan_42 wrote: post #88
aasasd wrote: post #74

Earlier quoted context omitted.

I'd guess extract the voice and then subtract it from the rest with something like Audacity. I'm not sure which operation would do that, but I believe that it exists. Also, other comments here speak of “separating into the voice and accompaniment,” so maybe the model/program already do exactly what you need.

Invert and mix. When you have an instrumental version of a song (from the same stems as the vocal version) this is already one way to get the vocals out without any fancy machine learning. The main tricks besides what you can do in Audacity like that are properly time-aligning the tracks (even if they drift a bit) and compensating for phase issues and compression. I wrote a dirty tool that does that and I've been mea…

If you don't mind sharing (even if it's cmd line), I'd love to explore.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#112

Post by jacquesm »

This is very timely. I've been working for about 3 months now on a utility that transforms mp3's to midi files. It's a hard problem and even though I'm making steady progress the end is nowhere in sight. This will give me something to benchmark against with for instance voice accompanied by piano. Thank you for making/posting this.

For an idea how this project is coming along:

https://jacquesmattheij.com/toccata.mp3

Yes, it's terrible :) This particular file the result of the following transformations:

midi file -> wav file (fluidsynth)

wav file -> midi file (my utility)

midi file -> wav file (fluidsynth once more)

wav file -> mp3 file (using lame)

Of course it also works for regular midi files (piano only for now). The reason why I use the workflow above is that it gives me a good idea how well the program works by comparing the original midi file with the output one.

But I did not yet have a way to deal with piano/voice which is a very common combination so this might really help me.

Possible applications: automatic music transcription, tutoring, giving regular pianos a midi 'out' port, using a regular piano as an arranger keyboard, instrument transformation and many others.

Having fun!

Edit: I've done a little write-up: https://jacquesmattheij.com/mp3-to-midi/

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#113

Post by semiotagonal »

unlinked_dll wrote: post #61
semiotagonal wrote: post #16

Earlier quoted context omitted.

Holy shit that works way better than I expected. The github project should link to this or a similar example, the technical description doesn't do it justice.

I really disagree. It sounds... awful. On par with other approaches, sure, but the main vocals sounds like a case study in digital artifacts and the accompaniment sounds like there's a filter automated over the track. Far from useful.

Do you know of something superior?

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#115

Post by jacquesm »

codedokode wrote: post #100
mwcampbell wrote: post #49

For your listening pleasure, here's a full-length demo. I decided to use the Jonathan Coulton classic "Re Your Brains", because I can legally share and modify his music under its Creative Commons license. First, the original: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Now the derived stems: Vocals: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Accompaniment: https://mwcampbell.u…

While it's a great technology, the result sounds somewhat robotic. On the original recording the voice sounds soft, but after separation it sounds like it is synthesized or passed through a vocoder, something is missing. The voice contains pieces of strumming sound. Guitar also sounds "blurred", as if someone cut an object from the picture and blurred the cut to make it less visible. Clap sound is distorted, on the o…

> While it's a great technology, the result sounds somewhat robotic.

That's like complaining about how bad the pig plays the violin. This is absolutely incredible. The complexity level for this problem is right off the scale and the software does a passable job of it. Given some time and more training data and a few more people working on it this has serious potential.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#117

Post by jacquesm »

jMyles wrote: post #48

Somewhat of a tangent, but does anyone have a recommendation of an open source (ideally python) program that can make MIDI from piano audio?

I've been working on this for the last 3 months.

Can you send me your audio file? I'll send you back the midi I can generate; it won't be perfect but it might be usable. See comment elsewhere in this thread.

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#118

Post by BrentOzar »

noja wrote: post #102
mwcampbell wrote: post #49

For your listening pleasure, here's a full-length demo. I decided to use the Jonathan Coulton classic "Re Your Brains", because I can legally share and modify his music under its Creative Commons license. First, the original: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Now the derived stems: Vocals: https://mwcampbell.us/tmp/spleeter-demo/jonathan-coulton-re-... Accompaniment: https://mwcampbell.u…

So Jonathan Coulton is now the new Suzanne Vega?

> So Jonathan Coulton is now the new Suzanne Vega?

Bravo. For people who didn't get the sublime reference, Suzanne Vega's song Tom's Diner was a benchmark test during development of the MP3.[1]

[1]: https://en.wikipedia.org/wiki/Tom%27s_Diner#The_"Mother_of_t...

Re: Spleeter: Extract voice, piano, drums, etc. from any music track

#120

Post by cartoonworld »

wpietri wrote: post #107
beachy wrote: post #39

Earlier quoted context omitted.

Rick says in one of his videos that he and some of his buddies have got old copies of the original source (separated) tracks, and they kind of pass them around between each other. I find that pretty amazing given the litigiousness of some in the music industry, but there we are. Side note: I discovered Rick Beato a few months ago and I've watched heaps of his videos. It's really fascinating hearing old classics torn…

Isn't that litigiousness mainly around money? Another important currency in the music world is respect. E.g., look at how Chamillionaire talks about Weird Al: http://yankovic.org/blog/2006/09/13/high-praise-from-chamill... Given that, I expect that a show titled "What Makes This Song Great" will do fine. Who doesn't love having somebody note the non-obviously good parts of their work. Especially if, as with Weird Al,…

The artists and performers are often quite reasonable. When you sign a major label deal, many sign away the rights to and control of their work in an effort to make a living and support their families. They need the money, also maybe a gold record.

Once they sign, the RIAA and label lawyers get to work, so the creator may not have any influence or own the masters.

Artists have a good chance of getting the point that authentic publicity is gonna garner authentic fans with authentic ticket stubs, but in the contract, on page 147 section 14a, under "Rights and Royalties" states ...

Post reply on HN