LOL
https://lamucal.ai/songs/john-coltrane/john-coltrane-giant-s...
It's definitely on to something. I wonder how would it perform if it was trained on Jazz.
Hybrid-Net: Real-time audio source separation, generate lyrics, chords, beat
21–30 of 67 posts
Re: Hybrid-Net: Real-time audio source separation, generate lyrics, chords, beat
#22Pretty cool! Tried it with RHCPs Dani California ( https://lamucal.ai/songs/red-hot-chili-peppers/dani-californ... ) and there's a lot of wrong chords still. Impressive nonetheless, and already quite useful in the song-part recognition (assuming it's all the ML)! Lyrics seem right too. The source separation only seems to be available when downloading their app, which I didn't do, so I can't comment on that.
I downloaded and tried their app, experiencing the audio source separation feature, and ended up with five tracks (piano, vocals, drums, bass, and others). It sounds pretty good, but unfortunately, there is no guitar track.
Re: Hybrid-Net: Real-time audio source separation, generate lyrics, chords, beat
#23Here's a question for folks who work on DL for audio: what are folks using for vocoders these days? I feel like that's where a lot of artifacts are introduced (at least for TTS) and the best methods a while ago were slow and autoregressive.
In recent years, there has been substantial advancement in vocoders for DL audio applications. WaveGAN and MelGAN have emerged as promising solutions, harnessing the power of generative adversarial networks (GANs) to produce high-fidelity audio. Furthermore, parallel-waveGAN and HiFi-GAN have showcased improved efficiency with quicker inference times while maintaining exceptional audio quality.
Thanks!!
Re: Hybrid-Net: Real-time audio source separation, generate lyrics, chords, beat
#24Hopefully no DMCA of something like that affect this project, long time ago I wanted to build a lyrics translation website, but apparently you may get in trouble for using copyrighted lyrics.
The data sources come from YouTube or user-uploaded audio, and the lyrics are extracted from the audio using AI models.
There appears to be an issue with the lyric matching, which I'm guessing is based on caching the previously generated lyrics for different songs with the same name. I noticed it specifically on the song "Sweet Pea" by the Tobasco Donkeys, where it shows tabs and lyrics but they don't match the song.
Re: Hybrid-Net: Real-time audio source separation, generate lyrics, chords, beat
#25So cool! I just tried it with a song with a fairly complicated chord progression - yesterday by the beatles. It did pretty well! But it got a couple parts wrong. Is there support for modifying the results of the chords/lyrics? I don't see it immediately.
We're about to roll out features for chord and lyric modifications, and we'll continue to optimize the model going forward.
Re: Hybrid-Net: Real-time audio source separation, generate lyrics, chords, beat
#26Any chance of adding documentation for running it locally? venv / requirements.txt at the very least, or a docker image?
Re: Hybrid-Net: Real-time audio source separation, generate lyrics, chords, beat
#27Tried it on a couple of my wife’s songs and she said it was quite inaccurate in terms of chords and tabs (the lyrics were pretty close though). This seems like one of those use cases where it’s not particularly useful until it gets above some minimum accuracy threshold.
Re: Hybrid-Net: Real-time audio source separation, generate lyrics, chords, beat
#28LOL https://lamucal.ai/songs/john-coltrane/john-coltrane-giant-s... It's definitely on to something. I wonder how would it perform if it was trained on Jazz.
Ha, Giant Steps was the first thing I tried as well. Not bad, certainly has some way to go.
Re: Hybrid-Net: Real-time audio source separation, generate lyrics, chords, beat
#29I tried the open source one that Spotify published a while ago on jazz trio music (just piano, double bass and drums) but it was pretty useless. My experiments with the trials of some commercial services, where you select an instrument and it extracts just that were much better.
If the piano is a Rhodes then extracting electric guitar works well, extracting a piano not at all.
Re: Hybrid-Net: Real-time audio source separation, generate lyrics, chords, beat
#30I tried it on one of my own tunes:
https://lamucal.ai/songs/adrian-holovaty/adrian-holovaty-the...
The beats/chords were consistently a full beat off, and the chords were probably only 50% right. I chose this tune because (to my ears) the harmony is pretty clear.
Compare this to my own manually created transcription of the same tune, and it's night-and-day difference:
https://www.soundslice.com/slices/tpbwc/
Beat detection and chord detection are hard problems, likely due to a lack of diverse training data. Chordify (another site that does this, which has been around for ages) has roughly similar performance.
Full disclosure: I run Soundslice, a website built around synced sheet music, in which there's no automatic transcription involved (maybe someday, but the tech isn't good enough yet!). I've been following these developments for 15+ years.