Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep Learning for Guitar Effect Emulation

teddykoker.com

Post reply on HN
Print view View original

11–20 of 168 posts

Re: Deep Learning for Guitar Effect Emulation

#11

Post by eru »

fab1an wrote: post #3

Pretty cool, though I wonder what the latency of this would be if used as a plugin? The author says it works in real-time, but to non music/audio folks this could mean '100 ms latency is real-time enough, right?' Generally, I think the audio VST business is a really fun space to be in for a lifestyle business, as it is way too small to be attractive for VCs. It seems like a space that provides many niches for lots of…

Real-time has a few slightly different meanings. So it's hard to say what the author means.

One meaning is just that you can guarantee specific deadlines. So if your programme can react within an hour guaranteed, that would be real-time. (Though usually we are talking about tighter deadlines, like what's needed to make ABS brakes work.)

For 'real time' music usage you wouldn't need strict guarantees, but something that's usually fast enough.

Re: Deep Learning for Guitar Effect Emulation

#12

Post by Tade0 »

jelling wrote: post #8

> We find that the model is able to reproduce a sound nearly indistinguishable from the real analog pedal. Maybe for the average person or buried in the mix, but the audio samples were easy to distinguish for me as a guitarist. The NN samples unnatural decay were a dead give away.

Also the pretty obvious quantization noise which sounds as if the effect had a wide bandwidth, which is impossible with their op-amps at these gains.

Re: Deep Learning for Guitar Effect Emulation

#13

Post by fxtentacle »

svantana wrote: post #5

End-to-end modelling is very enticing for the lazy engineer, unfortunately parameter control (knobs) are an important feature of most audio effects, and sampling enough of the parameter space will become prohibitive for more complex effects. That's why the traditional approach is divide-and-conquer. Also, I don't think this approach won't work well with time-varying effects such as chorus, although I'm happy to be pr…

Sadly, I believe you will be proven correct.

What that neural network learns is basically an approximation of a static impulse response. So while it can simulate linear time-invariant effects such as reverb quite nicely, it'll surely have issues with chorus.

Re: Deep Learning for Guitar Effect Emulation

#14

Post by sk0g »

fxtentacle wrote: post #13
svantana wrote: post #5

End-to-end modelling is very enticing for the lazy engineer, unfortunately parameter control (knobs) are an important feature of most audio effects, and sampling enough of the parameter space will become prohibitive for more complex effects. That's why the traditional approach is divide-and-conquer. Also, I don't think this approach won't work well with time-varying effects such as chorus, although I'm happy to be pr…

Sadly, I believe you will be proven correct. What that neural network learns is basically an approximation of a static impulse response. So while it can simulate linear time-invariant effects such as reverb quite nicely, it'll surely have issues with chorus.

Reverb is time invariant? You can set custom decay time, rate etc, so the one not can be heard for say, 10 seconds if you want to go full Devin Townsend. I'd think Chorus would work better.

I wanted to do a very similar project, but with an overdrive. Let's see if I get time anytime soon!

Re: Deep Learning for Guitar Effect Emulation

#15

Post by Tade0 »

Sounds great and I had to listen to both of the samples to guess correctly.

That being said the Tube Screamer is a somewhat simple effect: it's just a distortion with the clipping diodes moved to the feedback loop.

How possible would it be to get the famous A/B class amplifier voltage sag and associated changes in parameters of the whole amplifier, or in other words "will it chug"?

Re: Deep Learning for Guitar Effect Emulation

#16

Post by aea12 »

eru wrote: post #11
fab1an wrote: post #3

Pretty cool, though I wonder what the latency of this would be if used as a plugin? The author says it works in real-time, but to non music/audio folks this could mean '100 ms latency is real-time enough, right?' Generally, I think the audio VST business is a really fun space to be in for a lifestyle business, as it is way too small to be attractive for VCs. It seems like a space that provides many niches for lots of…

Real-time has a few slightly different meanings. So it's hard to say what the author means. One meaning is just that you can guarantee specific deadlines. So if your programme can react within an hour guaranteed, that would be real-time. (Though usually we are talking about tighter deadlines, like what's needed to make ABS brakes work.) For 'real time' music usage you wouldn't need strict guarantees, but something th…

Implementing a VST plugin is literally the exact definition of requiring strict latency guarantees. Your comment winds through a lot of unrelated comparisons to ultimately not make any sense.

“Usually fast enough” are three words that guarantee failure in a live show/MIDI environment, which is a large use case of VST and its peers beyond production. By extension, “usually fast enough” further guarantees nobody will ever use your software. That’s noticeable right away.

The question isn’t about compsci real-time theorycrafting, it’s “here’s a buffer of samples, if you don’t give it back in a dozen milliseconds the entire show collapses.” That’s pretty clearly meant by “real time“ contextually.

Re: Deep Learning for Guitar Effect Emulation

#17

Post by sailfast »

jelling wrote: post #8

> We find that the model is able to reproduce a sound nearly indistinguishable from the real analog pedal. Maybe for the average person or buried in the mix, but the audio samples were easy to distinguish for me as a guitarist. The NN samples unnatural decay were a dead give away.

Yeah - this was clearly audible on my phone speakers, especially during more muddy / multi-note sequences.

While it may not be able to emulate a real pedal to create one’s own sound, it would be interesting / fun for amateurs when applied as a post-filter with an interface that says “make this sound like X famous incredible track” coming out of a stock guitar signal.

Re: Deep Learning for Guitar Effect Emulation

#18

Post by phn »

svantana wrote: post #5

End-to-end modelling is very enticing for the lazy engineer, unfortunately parameter control (knobs) are an important feature of most audio effects, and sampling enough of the parameter space will become prohibitive for more complex effects. That's why the traditional approach is divide-and-conquer. Also, I don't think this approach won't work well with time-varying effects such as chorus, although I'm happy to be pr…

Even without parameterisation, it might be interesting as a "make my guitar sound like Jimmy Page" kind of tool.

Like you said, it will most likely have limitations, but it's still one more tool in the belt, regardless.

Re: Deep Learning for Guitar Effect Emulation

#19

Post by vnorilo »

sk0g wrote: post #14
fxtentacle wrote: post #13

Earlier quoted context omitted.

Sadly, I believe you will be proven correct. What that neural network learns is basically an approximation of a static impulse response. So while it can simulate linear time-invariant effects such as reverb quite nicely, it'll surely have issues with chorus.

Reverb is time invariant? You can set custom decay time, rate etc, so the one not can be heard for say, 10 seconds if you want to go full Devin Townsend. I'd think Chorus would work better. I wanted to do a very similar project, but with an overdrive. Let's see if I get time anytime soon!

Reverb is indeed linear time invariant (sans some rarer internal modulation techniques) but it's quite a high order filter.

Re: Deep Learning for Guitar Effect Emulation

#20

Post by InitialLastName »

sk0g wrote: post #14
fxtentacle wrote: post #13

Earlier quoted context omitted.

Sadly, I believe you will be proven correct. What that neural network learns is basically an approximation of a static impulse response. So while it can simulate linear time-invariant effects such as reverb quite nicely, it'll surely have issues with chorus.

Reverb is time invariant? You can set custom decay time, rate etc, so the one not can be heard for say, 10 seconds if you want to go full Devin Townsend. I'd think Chorus would work better. I wanted to do a very similar project, but with an overdrive. Let's see if I get time anytime soon!

>Reverb is time invariant?

You might want to familiarize yourself with [0]. Time-invariance is a specific property of a system, where the output (for any given input) has no dependency on if the input signal happens now or 1 second from now or 100 years from now (except for the corresponding delay). Most reverb models are, to a first approximation, time invariant, because the effect will have the same sound for the same guitar line, no matter when you play the line.

Chorus, on the other hand, has a (perhaps subtle) modulator to get that warbly (scientific word!) sound. It doesn't feel like a time-based effect, but it certainly is and that makes it quite a lot more difficult to mimic with a system that (as others have noted) boils down to an impulse response.

[0]https://en.wikipedia.org/wiki/Time-invariant_system

Post reply on HN