Live data from Hacker News

Show HN: Improved freemusicdemixer – AI music demixing in the browser

freemusicdemixer.com

1–10 of 42 posts

Show HN: Improved freemusicdemixer – AI music demixing in the browser

#1
Hi HN,

Last time I showed free-music-demixer, which people seemed to enjoy. It was a static website with a Javascript + WASM module to perform music demixing (or music source separation) using an AI model UMX-L (Open-Unmix) running client-side in the browser.

Since then, I have overhauled the project and made several improvements:

- The demixing/separation quality is higher now, since I implemented the missing post-processing step

- Memory usage is lower now by performing a custom segmented inference with a streaming LSTM, which should allow larger tracks (or, dare I say, arbitrarily-large tracks)

- There is a batch upload feature now to demix an entire folder of songs (and provide zip files of the stems)

- There are now dev logs printed to the website to show the progress better

Show HN: Improved freemusicdemixer – AI music demixing in the browser
freemusicdemixer.com

Re: Show HN: Improved freemusicdemixer – AI music demixing in the browser

#3
"Runs locally in your browser!

Unlike similar products, it’s free to use and doesn’t store your data. All processing is done in your browser, and your files are never uploaded anywhere. It runs well on computers and very slowly on smartphones; user beware."

I'm impressed by the fact that this was the choice made. I can only imagine it also helps keep the operational costs down, as well as liability for copyright and what not since they never become in possession of the content. However, it also means they "lose out" on a possible continuous source of training data which other less ethical evilCorp type companies would not pass up on

Re: Show HN: Improved freemusicdemixer – AI music demixing in the browser

#4
post #2

Is this the best possible demixing you can get? Or did you have to use a smaller / lower quality model to make it run in browsers?

In my first post, quite a lot of alternatives were discussed: https://news.ycombinator.com/item?id=36707877

The model I'm using is called Open-Unmix (https://github.com/sigsep/open-unmix-pytorch). In 2021, there was an update to Open-Unmix to include new weights, UMX-L, which made it perform better than it used to on the older weights (UMXHQ).

In the grand landscape of music demixing, I don't think UMX-L is near the top anymore.

_However_, the demixing performance of freemusicdemixer.com is very close to the full PyTorch performance of Open-Unmix UMX-L, despite the tricks I needed to get it working in the browser, such as splitting up the inference to operate on segments of the song, or making the LSTM operate on streaming segments rather than holding the entire track in the LSTM memory.

In my first release, I loaded and did inference on the entire track at once (like the PyTorch model), which frequently crashed or exceeded the 4GB WASM memory for medium or large-size tracks.

Re: Show HN: Improved freemusicdemixer – AI music demixing in the browser

#5
post #3

"Runs locally in your browser! Unlike similar products, it’s free to use and doesn’t store your data. All processing is done in your browser, and your files are never uploaded anywhere. It runs well on computers and very slowly on smartphones; user beware." I'm impressed by the fact that this was the choice made. I can only imagine it also helps keep the operational costs down, as well as liability for copyright and…

I'll be honest with you - I wouldn't even know what to do with user data if I collected it. I'm not great at training neural networks or understanding how to augment training with additional data.

Basically, the _only_ special thing about freemusicdemixer is that it runs client-side, because I'm better at writing C++ for a pre-trained network than I am at training new neural networks. However, it's a cool advantage so I'll keep promoting it as the distinguisher of my "product" (since I didn't create or train the underlying neural network, I just consider it a clever WASM frontend).

The operational costs are precisely $0 since it's a static website (aside from the domain registry).

Re: Show HN: Improved freemusicdemixer – AI music demixing in the browser

#7
post #6

For those interested, Facebook's Demucs page ( https://github.com/facebookresearch/demucs ) gives performance comparison for several models including open-unmix. See also: https://www.stemroller.com This runs as a local app on Windows and Mac.

My dream (and next major project) is to get Demucs (v4, Hybrid Transformer) running in WASM, similar to what I've done with Eigen/C++ and Open-Unmix.

Re: Show HN: Improved freemusicdemixer – AI music demixing in the browser

#9
post #4
post #2

Is this the best possible demixing you can get? Or did you have to use a smaller / lower quality model to make it run in browsers?

In my first post, quite a lot of alternatives were discussed: https://news.ycombinator.com/item?id=36707877 The model I'm using is called Open-Unmix ( https://github.com/sigsep/open-unmix-pytorch ). In 2021, there was an update to Open-Unmix to include new weights, UMX-L, which made it perform better than it used to on the older weights (UMXHQ). In the grand landscape of music demixing, I don't think UMX-L is near th…

It's super exciting to see people using open-unmix like this! I worked with the creator of the model to try to do the same as a university project! Our solution was... not great, to say the least, but I'm happy someone else managed to do it!

Re: Show HN: Improved freemusicdemixer – AI music demixing in the browser

#10
post #5
post #3

"Runs locally in your browser! Unlike similar products, it’s free to use and doesn’t store your data. All processing is done in your browser, and your files are never uploaded anywhere. It runs well on computers and very slowly on smartphones; user beware." I'm impressed by the fact that this was the choice made. I can only imagine it also helps keep the operational costs down, as well as liability for copyright and…

I'll be honest with you - I wouldn't even know what to do with user data if I collected it. I'm not great at training neural networks or understanding how to augment training with additional data. Basically, the _only_ special thing about freemusicdemixer is that it runs client-side, because I'm better at writing C++ for a pre-trained network than I am at training new neural networks. However, it's a cool advantage s…

Keep that mindset. Please. User data shouldn’t be your product. This is perfectly acceptable and thank you for doing this.
Post reply on HN