Pretty cool! Is this how Kemper amplifiers work when they do a capture?
[deleted]
Deep Learning for Guitar Effect Emulation
31–40 of 168 posts
Re: Deep Learning for Guitar Effect Emulation
#32End-to-end modelling is very enticing for the lazy engineer, unfortunately parameter control (knobs) are an important feature of most audio effects, and sampling enough of the parameter space will become prohibitive for more complex effects. That's why the traditional approach is divide-and-conquer. Also, I don't think this approach won't work well with time-varying effects such as chorus, although I'm happy to be pr…
Not saying that it is in any way practical or useful in the real world, but I think there are approaches which are more geared towards what we might understand as 'emulating' rather than modelling an effect. It seems that emulators can be learned from data with surprising efficiency [1]. These would be amenable to parameter control.
Re: Deep Learning for Guitar Effect Emulation
#33> We find that the model is able to reproduce a sound nearly indistinguishable from the real analog pedal. Maybe for the average person or buried in the mix, but the audio samples were easy to distinguish for me as a guitarist. The NN samples unnatural decay were a dead give away.
Even as a regular Joe it was easy for me to distinguish them, and though I was not very confident in my guess, I did guess correctly as well.
It was close though, so maybe for say a beginner on a shoe-string budget it would be perfectly acceptable.
Re: Deep Learning for Guitar Effect Emulation
#34Earlier quoted context omitted.
Implementing a VST plugin is literally the exact definition of requiring strict latency guarantees. Your comment winds through a lot of unrelated comparisons to ultimately not make any sense. “Usually fast enough” are three words that guarantee failure in a live show/MIDI environment, which is a large use case of VST and its peers beyond production. By extension, “usually fast enough” further guarantees nobody will e…
The show won't collapse, if you have one glitch an hour.
Yes, it absolutely 100% will, depending on what you mean by handwaving “glitch”. VST is built into chains, and a flaky plugin will derail an entire performance, often making downstream plugins crash. I’m speaking from extensive experience writing plugins and performing with them in multiple hosts and trigger setups. It’s not a robust protocol, but it gets the job done.
Are you speaking from some experience with which I’m unfamiliar where it’s okay for DSP code to fail hourly? Trying to understand your viewpoint.
Re: Deep Learning for Guitar Effect Emulation
#35Earlier quoted context omitted.
Not to mention if the inference is done on the CPU, it shouldn't be that hard to control it. The matrices are of a set size by the time you're running a VST; this is the actual simple answer. The medium answer is "this is a wavenet model, so inference is probably really expensive unless the continuous output is a huge improvement to performance".
Indeed. Having myself spent some time in the "VST lifestyle business" when I was in grad school (was selling a guitar emulation based on physical modelling synthesis), and now working in ML, I think there's no chance for such an approach to hit "mainstream" anytime soon. Even if you do your inference on CPU, most deep learning libraries are designed for throughput, not latency. In a VST plugin environment, you're als…
You might be able to combine it with the recent work on minimizing models to obtain something that is small enough to run reliably in real time.
Although the unusual structure of the net here may mean you're doing original and possibly publication-level work to adapt that stuff to this net structure.
If you were really interested in this, there could also be some profit in minimizing the model and then figuring out how to replicate it in a non-neural net way. Direct study of the resulting net may be profitable.
(I'm not in the ML field. I haven't seen anyone report this but I may just not be seeing it. But I'd be intrigued to see the result of running the size reduction on the net, running training on that network, then seeing if maybe you can reduce the resulting network again, then training that, and iterating until you either stop getting reduced sizes or the quality degrades too far. I've also wondered if there is something you could do to a net to encourage it not to have redundancies in it... although in this case the structure itself may do that job.)
Re: Deep Learning for Guitar Effect Emulation
#36End-to-end modelling is very enticing for the lazy engineer, unfortunately parameter control (knobs) are an important feature of most audio effects, and sampling enough of the parameter space will become prohibitive for more complex effects. That's why the traditional approach is divide-and-conquer. Also, I don't think this approach won't work well with time-varying effects such as chorus, although I'm happy to be pr…
Even without parameterisation, it might be interesting as a "make my guitar sound like Jimmy Page" kind of tool. Like you said, it will most likely have limitations, but it's still one more tool in the belt, regardless.
You're right in the "this is one more tool in the belt" sense but there are modelers like the Kemper and Fractal already out there that make your guitar sound like... anyone... and they are really convincing. I'd argue this is almost a solved problem. Still cool, nonetheless.
Re: Deep Learning for Guitar Effect Emulation
#37Earlier quoted context omitted.
Real-time has a few slightly different meanings. So it's hard to say what the author means. One meaning is just that you can guarantee specific deadlines. So if your programme can react within an hour guaranteed, that would be real-time. (Though usually we are talking about tighter deadlines, like what's needed to make ABS brakes work.) For 'real time' music usage you wouldn't need strict guarantees, but something th…
Implementing a VST plugin is literally the exact definition of requiring strict latency guarantees. Your comment winds through a lot of unrelated comparisons to ultimately not make any sense. “Usually fast enough” are three words that guarantee failure in a live show/MIDI environment, which is a large use case of VST and its peers beyond production. By extension, “usually fast enough” further guarantees nobody will e…
No, the other guy is right. Technically the definition of real-time can have a lot of leeway. Here's the paper linked in the article. Note how the authors never define what they really mean by real-time. They even make statements like "runs 1.9 times faster than real-time". They certainly imply your definition, but there's plenty of wiggle room to say "Well technically, I wasn't lying"
Re: Deep Learning for Guitar Effect Emulation
#38Earlier quoted context omitted.
Reverb is time invariant? You can set custom decay time, rate etc, so the one not can be heard for say, 10 seconds if you want to go full Devin Townsend. I'd think Chorus would work better. I wanted to do a very similar project, but with an overdrive. Let's see if I get time anytime soon!
Reverb is indeed linear time invariant (sans some rarer internal modulation techniques) but it's quite a high order filter.
Ah righto, the reverb pedal I'm most familiar with turns out to not be just reverb - EQD Afterneath does a whole bunch of funky stuff. Plain reverb though, yeah. I was approaching this more from the angle of training a neural network, where the input and output waves have to be correlated over a great span of time/ samples.
Re: Deep Learning for Guitar Effect Emulation
#39Pretty cool, though I wonder what the latency of this would be if used as a plugin? The author says it works in real-time, but to non music/audio folks this could mean '100 ms latency is real-time enough, right?' Generally, I think the audio VST business is a really fun space to be in for a lifestyle business, as it is way too small to be attractive for VCs. It seems like a space that provides many niches for lots of…
Do solo or small shop vst plugin developers make any money?
I’m curious if anyone has any direct knowledge about that.
There are so many professional activities similar to that where no one makes any money and people really just do it for the love, and then there are seemingly similar things like that where people make surprisingly large amounts of money.
Re: Deep Learning for Guitar Effect Emulation
#40Earlier quoted context omitted.
Real-time has a few slightly different meanings. So it's hard to say what the author means. One meaning is just that you can guarantee specific deadlines. So if your programme can react within an hour guaranteed, that would be real-time. (Though usually we are talking about tighter deadlines, like what's needed to make ABS brakes work.) For 'real time' music usage you wouldn't need strict guarantees, but something th…
Implementing a VST plugin is literally the exact definition of requiring strict latency guarantees. Your comment winds through a lot of unrelated comparisons to ultimately not make any sense. “Usually fast enough” are three words that guarantee failure in a live show/MIDI environment, which is a large use case of VST and its peers beyond production. By extension, “usually fast enough” further guarantees nobody will e…
"Usually fast enough" is unfortunately the only guarantee a preemptive multitasking OS can give you. Unless your system is guaranteeing your program x cycles of uninterrupted processing per frame of audio and you can consistently process the frame in that amount of cycles, the only mitigation is to deliver frames in large enough chunks that you never run out of time in practice under agreeable circumstances.
That said, I agree that the question of what "real-time" might mean is irrelevant given the context.