Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep Learning for Guitar Effect Emulation

teddykoker.com

Post reply on HN
Print view View original

51–60 of 168 posts

Re: Deep Learning for Guitar Effect Emulation

#51

Post by luckydata »

fxtentacle wrote: post #13
svantana wrote: post #5

End-to-end modelling is very enticing for the lazy engineer, unfortunately parameter control (knobs) are an important feature of most audio effects, and sampling enough of the parameter space will become prohibitive for more complex effects. That's why the traditional approach is divide-and-conquer. Also, I don't think this approach won't work well with time-varying effects such as chorus, although I'm happy to be pr…

Sadly, I believe you will be proven correct. What that neural network learns is basically an approximation of a static impulse response. So while it can simulate linear time-invariant effects such as reverb quite nicely, it'll surely have issues with chorus.

Then what you need is a multi dimensional matrix of impulse responses, one for each combination of parameter. You can further simplify the model by limiting to only useful combination ranges etc...

Re: Deep Learning for Guitar Effect Emulation

#52

Post by [deleted] »

fab1an wrote: post #3

Pretty cool, though I wonder what the latency of this would be if used as a plugin? The author says it works in real-time, but to non music/audio folks this could mean '100 ms latency is real-time enough, right?' Generally, I think the audio VST business is a really fun space to be in for a lifestyle business, as it is way too small to be attractive for VCs. It seems like a space that provides many niches for lots of…

[deleted]

Re: Deep Learning for Guitar Effect Emulation

#53

Post by [deleted] »

swebs wrote: post #37
aea12 wrote: post #16

Earlier quoted context omitted.

Implementing a VST plugin is literally the exact definition of requiring strict latency guarantees. Your comment winds through a lot of unrelated comparisons to ultimately not make any sense. “Usually fast enough” are three words that guarantee failure in a live show/MIDI environment, which is a large use case of VST and its peers beyond production. By extension, “usually fast enough” further guarantees nobody will e…

No, the other guy is right. Technically the definition of real-time can have a lot of leeway. Here's the paper linked in the article. Note how the authors never define what they really mean by real-time. They even make statements like "runs 1.9 times faster than real-time". They certainly imply your definition, but there's plenty of wiggle room to say "Well technically, I wasn't lying" https://www.mdpi.com/2076-3417/…

[deleted]

Re: Deep Learning for Guitar Effect Emulation

#54

Post by luckydata »

tricky wrote: post #36
phn wrote: post #18

Earlier quoted context omitted.

Even without parameterisation, it might be interesting as a "make my guitar sound like Jimmy Page" kind of tool. Like you said, it will most likely have limitations, but it's still one more tool in the belt, regardless.

You're right in the "this is one more tool in the belt" sense but there are modelers like the Kemper and Fractal already out there that make your guitar sound like... anyone... and they are really convincing. I'd argue this is almost a solved problem. Still cool, nonetheless.

Building the model is not solved though. Kemper sort of does that by not sure what, but an approach that simply measures the effect and creates a complete model in hours would radically change the industry. Companies like Yamaha (line6) would be able to add hundreds of simulations in months instead of a couple.

Re: Deep Learning for Guitar Effect Emulation

#55

Post by kleer001 »

moralestapia wrote: post #50
ff7f00 wrote: post #48

Earlier quoted context omitted.

There are definitely big players making a lot of money from plugins they develop. Here are a few to check out: ($1200) https://www.native-instruments.com/en/products/komplete/bund... ($300) https://www.soundtoys.com/product/soundtoys-5/ ($500) https://www.arturia.com/products/analog-classics/v-collectio... However, piracy is also pretty big when it comes to plugins.

Sure, but, >Do solo or small shop vst plugin developers make any money?

The implication seems to be "no".

Re: Deep Learning for Guitar Effect Emulation

#56

Post by zwieback »

jelling wrote: post #8

> We find that the model is able to reproduce a sound nearly indistinguishable from the real analog pedal. Maybe for the average person or buried in the mix, but the audio samples were easy to distinguish for me as a guitarist. The NN samples unnatural decay were a dead give away.

Yeah, real pedal sounds much "better" but maybe we're just used to how they sound.

Re: Deep Learning for Guitar Effect Emulation

#57

Post by TrackerFF »

Isn't this essentially just learning the case of learning one function, with set parameters?

I.e, if you want to build a complete model of the tubescreamer, you'd essentially have to train a model for each possible setting on the pedal - or in other words, every combination of the knobs.

Sounds like a real chore, if you were to actually do that physically - and in the end, don't you just want to learn the impulse response of the circuit?

I know some tools - like the Kemper modelling gear, are made for that exact purpose, and with extremely convincing results.

Re: Deep Learning for Guitar Effect Emulation

#58

Post by samplenoise »

fab1an wrote: post #3

Pretty cool, though I wonder what the latency of this would be if used as a plugin? The author says it works in real-time, but to non music/audio folks this could mean '100 ms latency is real-time enough, right?' Generally, I think the audio VST business is a really fun space to be in for a lifestyle business, as it is way too small to be attractive for VCs. It seems like a space that provides many niches for lots of…

There's latency and there's the somewhat separate question of how much time is needed to make a prediction. Wavenet is causal (no look-ahead) and operates on the sample level so there are no buffers and thus no latency in the strict sense, beyond encoding/decoding into the sample rate and format required by the ML model, which should take <1ms. Whether a model manages to make a prediction in that amount of time depends on things like the receptive field and number of layers. The linked paper says their custom implementation runs at 1.1x real-time. I guess this isn't impossible; their receptive field is ~40ms, vs. 300 for the original (notoriously slow) wavenet, and the model is likely to have less layers and channels.

Re: Deep Learning for Guitar Effect Emulation

#59

Post by Scene_Cast2 »

TrackerFF wrote: post #57

Isn't this essentially just learning the case of learning one function, with set parameters? I.e, if you want to build a complete model of the tubescreamer, you'd essentially have to train a model for each possible setting on the pedal - or in other words, every combination of the knobs. Sounds like a real chore, if you were to actually do that physically - and in the end, don't you just want to learn the impulse res…

Not quite. As long as the knobs make consistent changes, just feed some large amount of tests and the model should generalize (smartly interpolate) the rest.

What I do have a problem with is that if the pedal is already implemented digitally, then all the human interpretability, along with the classic DSP machinery, is thrown out the window. A better approach would be to build the pedal via a differentiable programming language and then try to gradient descent toward some analog "can't get this juicy tube sound digitally" variant.

Re: Deep Learning for Guitar Effect Emulation

#60

Post by boomlinde »

aea12 wrote: post #41
boomlinde wrote: post #40

Earlier quoted context omitted.

"Usually fast enough" is unfortunately the only guarantee a preemptive multitasking OS can give you. Unless your system is guaranteeing your program x cycles of uninterrupted processing per frame of audio and you can consistently process the frame in that amount of cycles, the only mitigation is to deliver frames in large enough chunks that you never run out of time in practice under agreeable circumstances. That sai…

It is completely irrelevant, given the context. The only, only, only thing real-time means here is “can be run on a live signal passing through it” rather than “is a slow, offline effect for a DAW”. No hard real-time, no soft real-time, no QNX, no pulling out the college compsci textbook. There IS real-time in that sense in DSP, it just isn’t in a VST plugin. I’ll repeat again that any compsci theorycrafting is not t…

Just to be perfectly clear here because I'm not sure you're just using my post as a soapbox or if you have misunderstood my argument: I agree that it's clear what real-time means in this context. I disagree that "usually fast enough" guarantees failure for a VST, because in the case of VST, "usually fast enough" is the only guarantee the host operating system will offer your software.

It's not "theorycrafting" to say that real-time music software running in a preemptive multitasking operating system without deterministic process time allocation will have to suffer the possibility of occasional drops. It happens in practice and audio drivers have to be implemented to account for the bulk of it, and the VST API is designed in such a way that failure to fill a buffer on time needn't be fatal.

Post reply on HN