[deleted]
Real-Time Noise Suppression Using Deep Learning
61–70 of 73 posts
Re: Real-Time Noise Suppression Using Deep Learning
#62Earlier quoted context omitted.
Disclosure: I'm the author of the blog post and co-founder at 2Hz. This is a guest post on NVIDIA Developer Blog. The author of the technology is a startup called 2Hz (2hz.ai). Our passion is to improve voice audio quality in audio/video calls. It's a tough problem but also fun to work on. Agree, breathing, reverb, noise are all problems and should be fixed. We started with noise and already shipped a product you can…
Impressive results! Any plans for a windows/linux version? Does it also cancel noise on incoming audio?
Windows will come soon. Linux - no plans yet.
Yes, it cleans the incoming as well.
Re: Real-Time Noise Suppression Using Deep Learning
#63This is amazing! Full props to the Nvidia team that accomplished this. I downloaded the Mac app they provided [1] which I highly suggest everyone with a mac tests out. I ran it on my old MacBook Air 2013 using daily.co. It worked like a charm. Definitely using this in the next group chat, where there is always someone who forgets to turn off their microphone. One cool side effect is that it actually removes the rever…
copied from another comment, NOT MINE:
COMMENT FOLLOWS
davitb 18 hours ago
Disclosure: I'm the author of the blog post and co-founder at 2Hz. This is a guest post on NVIDIA Developer Blog. The author of the technology is a startup called 2Hz (2hz.ai). Our passion is to improve voice audio quality in audio/video calls. It's a tough problem but also fun to work on.
Agree, breathing, reverb, noise are all problems and should be fixed. We started with noise and already shipped a product you can try on your Mac. The app is called Krisp (krisp.ai).
Reverb, breathing, voice cutting will come next.
Re: Real-Time Noise Suppression Using Deep Learning
#64What's wrong with using multiple microphones? A mic element costs about thirty cents, and the processing power required for noise cancellation already exists in the CPU of the mobile device. I think it's particularly interesting that Amazon has made microphe arrays particularly cheap, due to Alexa. MiniDSP offers a microphone array for under $100, which is an unheard of price considering what these cost ten years ago…
Apple sold over 200million iPhones last year. That's 60million$ in saved BOM costs. At the scale that smartphones are sold, saving pennies here and there adds up to millions in additional revenue.
Obviously there's costs to running servers somewhere, but that hasn't stopped companies from making similar decisions for a variety of other services.
Re: Real-Time Noise Suppression Using Deep Learning
#65What's wrong with using multiple microphones? A mic element costs about thirty cents, and the processing power required for noise cancellation already exists in the CPU of the mobile device. I think it's particularly interesting that Amazon has made microphe arrays particularly cheap, due to Alexa. MiniDSP offers a microphone array for under $100, which is an unheard of price considering what these cost ten years ago…
Isn’t that addressed in the article that multi microphone works when there is physical distance between the mics — on a phone that’s around 4 inches.
What is that on a watch? An inch? Maybe one can be put on the band. Also on a phone the other mic is on the opposite side of the phone so it isn’t directly in the line of fire from your voice.
Re: Real-Time Noise Suppression Using Deep Learning
#66What's wrong with using multiple microphones? A mic element costs about thirty cents, and the processing power required for noise cancellation already exists in the CPU of the mobile device. I think it's particularly interesting that Amazon has made microphe arrays particularly cheap, due to Alexa. MiniDSP offers a microphone array for under $100, which is an unheard of price considering what these cost ten years ago…
How does multi microphone filtering work? I guess they localize different sound sources by cross-correlation (to get the timings) and triangulation (based on the timings and the speed of sound)?
It's pretty neat! Here's how it works:
When you record with a single microphone, you are going to pick up a great deal of background noise. This is because the mic will pic up the person speaking AND the background noise; there's no way to differentiate the two.
With two microphones, we know the following:
1) we know where the microphones are
2) we have a general idea where the persons mouth is, because we know how they hold the phone
Based on that, we have a good idea of how long it should take for the sound to arrive, because the speed of sound is a fixed number.
The first time I ever heard a dual mic phone was when one of my coworkers made a call from the inside of our data center. Typically, he'd have to shout into the phone, because the data center was so noisy, and worst of all, the noise was completely random and broadband. But with dual mics, poof, background noise is gone. It was almost like he was speaking in a quiet room.
Amazon Alexa takes this quite a bit further, and uses something called "beamforming." What beamforming allows you to do is to determine WHERE the person is in the room, based on the arrival times of the sound. It's sort of the inverse of a dual mic setup; in a dual mic setup we can 'clean up' the signal because we know where the person speaking is. In a beamforming arrangement, we can use the arrival times to FIGURE OUT where the person is in the room.
If some security company was clever, they could probably use a beamforming microphone array to train a camera on people in the room.
And keep in mind, Alexa beamforming is two dimensional, but you could go crazy and do a 3D beamforming array if you wanted to! (Alex only knows where you are on a horizontal plane.)
Re: Real-Time Noise Suppression Using Deep Learning
#67Earlier quoted context omitted.
How does multi microphone filtering work? I guess they localize different sound sources by cross-correlation (to get the timings) and triangulation (based on the timings and the speed of sound)?
I think the (or perhaps only one) key phrase is "beamforming". A single microphone element has a certain sensitivity pattern (e.g. it may be a very directional microphone, or be equally sensitive in all directions). With multiple pick-ups, you can emulate some different sensitivity patterns. A related idea in radar is synthetic-aperture radar (SAR).
Great point.
A lot of the interesting things in audio were inspired by radar. Dan Wiggins at Sonos used to work on radar, and Don Keele created a loudspeaker technology called "CBT" that's based on radar technology.
Because microphones are basically the inverse of loudspeakers, what works in loudspeaker arrays can also work in microphone arrays.
Re: Real-Time Noise Suppression Using Deep Learning
#68I downloaded the mac app, configured a virtual device to send the system output to the "Krisp Speaker" and verified that it cuts most of the music out of what I'm listening to, leaving only the voice (at a some what degraded quality). I wish I could configure it _cancel_ ambient noise, not just remove it from the input signal.
In your perception, what is the difference between "cancelling" a signal and removing it?
Phase Cancellation [1]
[1] https://www.sageaudio.com/blog/pre-mastering-tips/phase-canc...
Re: Real-Time Noise Suppression Using Deep Learning
#69Love it. Don’t really love the idea of audio contents of conversations being routed to a cloud server for processing though — needs to stay on-device for privacy.
Their ultimate goal must be to be acquired by Apple, Google, or similar. This will never fly as a third-party app/install, even if it's "promised" to be on-device (such promises can change). Moreover, the average user isn't going to know about, care about, or seek out such an app, let alone pay for it. There's no widespread reach or profit in selling direct to consumer. The only way this works is for it to be built i…
Agree, I wasn't suggesting users would download a special app to use it - I'd like to see the tech make its way into the OS for all voice input.
I can see why the cloud processing makes sense for certain applications / licensers / acquirers (e.g. a VoIP provider like Xoom), but voice comms is really the domain of smartphones, and my hunch is most are plenty powerful enough to do this processing locally.
Re: Real-Time Noise Suppression Using Deep Learning
#70Love it. Don’t really love the idea of audio contents of conversations being routed to a cloud server for processing though — needs to stay on-device for privacy.
This technology is already integrated into Krisp app ( https://krisp.ai ) and it runs all locally on device.
Thanks for the heads up! I really like what you're doing - not only is it great for the general public, it's a game changer for people with difficulties hearing.