Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM

zhinit.dev

Post reply on HN
Print view View original

71–80 of 93 posts

Re: How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM

#71

Post by BrandNewRetro »

nxpnsv wrote: post #60
thangalin wrote: post #41

Slightly off-topic. Now that 1920s jazz music is falling into public domain, has anyone tried to reinvigorate the music using AI and generative adversarial approaches? Pre-1940s music didn't have high-fidelity sound, so the strong bass lines weren't captured. In theory, we could "downgrade" modern recordings to sound like 1920s recordings, then use adversarial techniques to train the machine on how to restore the ant…

There are plenty of current human bands that play this music really well though...

Just giggling silently as I imagine the stereotypical trad jazz purist's response to OP. Man, those guys are... Uhhh... Loyal to that vision of music.

Re: How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM

#72

Post by BrandNewRetro »

I have three (and pray I do not come up with any more) needs for AI audio apps.

One, a ubiquitous restoration model. Find degraded copies of music in the wild, old YouTube's, transcodes, vinyl rips, bad masters, half destroyed tapes... Pair them with modern pristine lossless encodes of the same music, train. Then use that model on music we don't have pristine copies of.

The second is similar but more specific. There are so many stems floating around from popular music. My idea is to compare individual stems against the results from MVS/Spleeter(same song, same instrument). This would surely stand a chance of pushing that tech forward, so we can treat the FFT artefact heavy sound of new efforts.

Thirdly, from a creative point of view, I wanna do the equivalent of image to image on my tracks... But I actually want it to hallucinate in the manner of the early deep dream images, I want to be able to play with that space..

I can knock out musak to spek in minutes already, gen music is just reducing low effort to nearly no effort, preventing people with needs from networking with creators.... Uhh.. but I think that's a very general issue with Gen AI away from the corporate/entrepreneurial dev space

Re: How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM

#73

Post by drcongo »

tunk wrote: post #57
zhinit wrote: post #45

Earlier quoted context omitted.

Interesting! I had not seen this. On their website they mention diffusion but not the other models so it might not be identical but its definitely similar.

That's fair :) I also could've phrased my comment a bit more politely.

It reminded me of the (possibly apocryphal) story about Liam Gallagher trashing a hotel room on tour - when asked by a reporter "why? It's been done before." he supposedly replied "yeah, but not by me".

Sometimes the "by me" is the interesting / fun / instructive part.

Re: How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM

#75

Post by tgv »

pennomi wrote: post #13
tgv wrote: post #5

Earlier quoted context omitted.

Decomposing sounds from (fully produced?) tracks into underlying components, and then giving the user the option to synthesize them with different parameter settings. I think.

I was trying (and failing) to do this the other day. It’s a really interesting problem space and I love to see someone with a more solid foundation give it a try.

There is better software, although it might not meet OP's goal. There's e.g. Melodyine, which can identify (and change) notes from individual instruments in polyphonic passages, and there are also tools to estimate and remove reverb and other forms of processing. Those are based on classical DSP. OP just wanted to use "AI".

Re: How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM

#77

Post by mk_stjames »

This is cool, I had some questions because I was trying to understand this more - not the diffusion model training part, because that I've actually seen before- but the actual way you bundled this into a web application and the choices made...

Did you publish the actual trained model anywhere? I see how in the code there is python for how individual samples can be generated, but the model checkpoint pulldown comes from a directory that... I don't see.

I then went through the code of how this runs on the web and- I'm not a web dev guy- so I'm pretty confused at all the bits bolted together to make this into a web app. It seems like there is a WASM bit that is compiled from a typical C++ audio plugin that is doing the stuff like conv reverb and limiting and distortion in the web app - all that is oldschool, non-generative AI, DSP being applied to the samples. Then the samples are just... a few default generated samples, to start- where are they pulled from, physically? And you have a login requirement to spool up the actual generative AI part to generate new samples to run into the DSP (because that needs a GPU on the backend to do, so, a login to help rate limit this)

How big is the actual generative model? Did you ever think about building the generation engine into the WASM bundle, using maybe WebGPU in the WASM to accelerate in a platform agnostic way, so that the entire app would run offline in someone's browser window?

I'm having fun just playing with the kick program without a login, which, again, am I right in saying in that mode there is no gen AI of sampled happening server side, it is just playing starting with some pre-made samples?

Re: How to Train a Gen AI Kick Drum Model on Your Old Linux Desktop with 6GB VRAM

#78

Post by codedokode »

What about copyright? Can one take 13,000 copyrighted works, train the model and release it for free so that nobody ever buys any of those 13,000 samples? Can one (legally) convert all commercial sample libraries into a free neural network?
Post reply on HN