Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM

zhinit.dev

Post reply on HN
Print view View original

81–90 of 93 posts

Re: How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM

#81

Post by zhinit »

codedokode wrote: post #78

What about copyright? Can one take 13,000 copyrighted works, train the model and release it for free so that nobody ever buys any of those 13,000 samples? Can one (legally) convert all commercial sample libraries into a free neural network?

Im not selling a product here so I'm not going to worry about it for my purposes.

This is the hot topic in AI ethics.

Is it just learning the same way that a human learns from hearing songs on the radio and playing them, or is it just a compression algorithm? There is not a clear answer here but it seems like the legal system so far is letting things slide and agreeing with the former argument.

(This can also be applied to other areas like open AI reading the NYT)

Re: How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM

#82

Post by zhinit »

mk_stjames wrote: post #77

This is cool, I had some questions because I was trying to understand this more - not the diffusion model training part, because that I've actually seen before- but the actual way you bundled this into a web application and the choices made... Did you publish the actual trained model anywhere? I see how in the code there is python for how individual samples can be generated, but the model checkpoint pulldown comes fr…

The weights are ~300MB and are on hugging face here. https://huggingface.co/zhinit/kick-gen-v1

It was a lot of work to get a good DSP to work on the web hahah. Yes, I am writing the DSP in C++ and compiling to WASM. Im using multithreading so the audio work is done in the AudioWorklet while the UI is run in the main thread. I was thinking of writing another article on this because it's pretty interesting and a bit complex.

I sound designed some of the kicks myself and some of them are from sample packs. I just renamed them all to have German names.

If I wanted the model to run completely in the frontend I would need the user to download 300MB of weights and it would probably be tricky making sure it works on everyone's hardware. So I though about this but it didn't seem like the best option. I'm pretty sure it is possible though.

And yes I put rate limiting so no one goes crazy on my credits.

Im glad you're enjoying it!

Re: How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM

#83

Post by zhinit »

BrandNewRetro wrote: post #72

I have three (and pray I do not come up with any more) needs for AI audio apps. One, a ubiquitous restoration model. Find degraded copies of music in the wild, old YouTube's, transcodes, vinyl rips, bad masters, half destroyed tapes... Pair them with modern pristine lossless encodes of the same music, train. Then use that model on music we don't have pristine copies of. The second is similar but more specific. There…

For 1) another commenter had a simmilar idea. I think a GAN would be able to do this. It's just a matter of collecting the data

For 2) SUNO's stem separation is pretty good. The open sources ones (like spleeter) are also not bad, but they are pretty hit or miss.

For 3) This is a really cool idea. The deep dream images produced the fur and eyeball textures because they used imagenet data which had a bunch of dogs and other animals in it. The trick would be finding a good data set to use for this. May be there is a public source of animal sounds floating around you could try this with.

And yeah think people 100% generating songs is pretty wack, but I could see why it would be fun to someone who is not a musician.

Re: How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM

#87

Post by thenthenthen »

Couldnt get sound to work on iOS but back at my Desktop now and wow these examples/presets sound really cool, great job! (The examples in the write up sound very weird btw...). Try out the 'app' here > https://kick-with-reverb.vercel.app/

Re: How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM

#89

Post by thangalin »

wildzzz wrote: post #54
thangalin wrote: post #41

Slightly off-topic. Now that 1920s jazz music is falling into public domain, has anyone tried to reinvigorate the music using AI and generative adversarial approaches? Pre-1940s music didn't have high-fidelity sound, so the strong bass lines weren't captured. In theory, we could "downgrade" modern recordings to sound like 1920s recordings, then use adversarial techniques to train the machine on how to restore the ant…

It might be easier than that. Are the bass lines totally missing or are they just very weak? If you can capture a recording using vintage equipment and the placing of it, you can get the system response. Run the original recordings through an inversion of the response and you should get really close. Another possible method is to find the transform between an identical modern recording of the song and use the differe…

Some examples:

The Mooche

* https://www.youtube.com/watch?v=BPJ5vNmVL3I (The Duke, 1928)

* https://www.youtube.com/watch?v=tLdCq2PKM4o (John Barry, 1984)

Sugar Foot Stomp

* https://www.youtube.com/watch?v=qEdIWVsfPXs (Henderson, 1925)

* https://www.youtube.com/watch?v=35foefTrdLo (Nighthawks, 1993)

Doctor Jazz

* https://www.youtube.com/watch?v=HTYAaX7lqjQ (Morton, 1926)

* https://www.shazam.com/song/1687138286/doctor-jazz (Asaro & Fat Babies, 2016)

I was wondering if anyone was working on this (i.e., using pure software). Period equipment, real musicians, and such would be time consuming, arduous, and cost prohibitive.

Re: How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM

#90

Post by zhinit »

thenthenthen wrote: post #87

Couldnt get sound to work on iOS but back at my Desktop now and wow these examples/presets sound really cool, great job! (The examples in the write up sound very weird btw...). Try out the 'app' here > https://kick-with-reverb.vercel.app/

Yeah, for web apps you have to make sure your phone is not on silent and the ringer volume is up. This is why most audio apps for phones will force you to use their native app.

I was looking into a way around this and I don't think it exists without building out an entire phone app.

Post reply on HN