Live data from Hacker News

Real-Time Noise Suppression Using Deep Learning

devblogs.nvidia.com

41–50 of 73 posts

Re: Real-Time Noise Suppression Using Deep Learning

#41
What's wrong with using multiple microphones?

A mic element costs about thirty cents, and the processing power required for noise cancellation already exists in the CPU of the mobile device.

I think it's particularly interesting that Amazon has made microphe arrays particularly cheap, due to Alexa. MiniDSP offers a microphone array for under $100, which is an unheard of price considering what these cost ten years ago.

https://www.minidsp.com/products/usb-audio-interface/uma-8-m...

Re: Real-Time Noise Suppression Using Deep Learning

#42

What's wrong with using multiple microphones? A mic element costs about thirty cents, and the processing power required for noise cancellation already exists in the CPU of the mobile device. I think it's particularly interesting that Amazon has made microphe arrays particularly cheap, due to Alexa. MiniDSP offers a microphone array for under $100, which is an unheard of price considering what these cost ten years ago…

How does multi microphone filtering work? I guess they localize different sound sources by cross-correlation (to get the timings) and triangulation (based on the timings and the speed of sound)?

Re: Real-Time Noise Suppression Using Deep Learning

#43
post #14

Does this deep learning noise cancelling also work for music with headphones? If so then we can ditch proprietary noise cancelling headphones and just use the phones?

What others haven't mentioned and I think is important is that active noise canceling headphones don't try modify your incoming audio. They listen to the outside environment and try to cancel external audio that you'd otherwise hear. For example, listening to a podcast on a noisy bus ride.

This sort of noise canceling tries to remove the unwanted noise that is already mixed in with the wanted sound in the same recording. For example, recording a podcast on a noisy bus ride.

Re: Real-Time Noise Suppression Using Deep Learning

#44
post #16

Earlier quoted context omitted.

Disclosure: I'm the author of the blog post and co-founder at 2Hz. This is a guest post on NVIDIA Developer Blog. The author of the technology is a startup called 2Hz (2hz.ai). Our passion is to improve voice audio quality in audio/video calls. It's a tough problem but also fun to work on. Agree, breathing, reverb, noise are all problems and should be fixed. We started with noise and already shipped a product you can…

Hi! As someone who seems to struggle more than most to understand people on video calls, I'd like to give you my impressions. Something struck me about the sample video. The very first sample included background noise, but it was very easy to understand regardless of the noise, probably because it was recorded by a pro microphone rather than a phone. Every other sample was far more difficult, regardless of noise remo…

> I don't know how representative my experience is.

As someone who works with speech content, this seems unusual. Typically, low frequencies are reduced because there's not much useful voice signal there—for example, NPR typically rolls off frequencies below 250 Hz.

Re: Real-Time Noise Suppression Using Deep Learning

#45

What's wrong with using multiple microphones? A mic element costs about thirty cents, and the processing power required for noise cancellation already exists in the CPU of the mobile device. I think it's particularly interesting that Amazon has made microphe arrays particularly cheap, due to Alexa. MiniDSP offers a microphone array for under $100, which is an unheard of price considering what these cost ten years ago…

How does multi microphone filtering work? I guess they localize different sound sources by cross-correlation (to get the timings) and triangulation (based on the timings and the speed of sound)?

I think the (or perhaps only one) key phrase is "beamforming". A single microphone element has a certain sensitivity pattern (e.g. it may be a very directional microphone, or be equally sensitive in all directions). With multiple pick-ups, you can emulate some different sensitivity patterns.

A related idea in radar is synthetic-aperture radar (SAR).

Re: Real-Time Noise Suppression Using Deep Learning

#46

Earlier quoted context omitted.

Hi! As someone who seems to struggle more than most to understand people on video calls, I'd like to give you my impressions. Something struck me about the sample video. The very first sample included background noise, but it was very easy to understand regardless of the noise, probably because it was recorded by a pro microphone rather than a phone. Every other sample was far more difficult, regardless of noise remo…

> I don't know how representative my experience is. As someone who works with speech content, this seems unusual. Typically, low frequencies are reduced because there's not much useful voice signal there—for example, NPR typically rolls off frequencies below 250 Hz.

Thanks for your viewpoint!

Here's something concrete: the first phrase in the video ends with "small demonstration", but starting with the second instance, I distinctly hear "sall" instead of "small". In the version with the noise, the "m" sounds like an aberration of the noise and is detectable. With the noise removed, the "m" is replaced with a blip that sounds like an encoding error.

Re: Real-Time Noise Suppression Using Deep Learning

#47

I am really impressed with what Nvidia is doing here. I think there is a huge market for improving sound quality in video calls. For me, roughly every second call I make is somehow harmed by some kind of "bad audio" problems. Breathing, reverb, noise, clipping, too silent, there are so many things that can go wrong. And this really harms the productivity of video calls. I have started collecting and building tools to…

And don't forget hearing aids. That's a market that's only going to keep getting bigger over time. The first people to ship a super-power-efficient ASIC for wind/restaurant denoising which allows reasonable hearing aid battery life are going to make a well-deserved fortune.

Re: Real-Time Noise Suppression Using Deep Learning

#48

What's wrong with using multiple microphones? A mic element costs about thirty cents, and the processing power required for noise cancellation already exists in the CPU of the mobile device. I think it's particularly interesting that Amazon has made microphe arrays particularly cheap, due to Alexa. MiniDSP offers a microphone array for under $100, which is an unheard of price considering what these cost ten years ago…

The Matrix Creator has an 8 mic array as well plus a bunch of other hardware goodies for about the same price. No experience with it but it looks cool,

https://www.matrix.one/products/creator

Re: Real-Time Noise Suppression Using Deep Learning

#49
post #26
post #13

Really impressive results, though I wish they had gone more into the deep learning part of it (but I guess that's probably the secret sauce). Can't help but notice how well Nvidia is positioned for what appears to be a growing wave of demand for GPUs. Surprised this hasn't reflected in their share price (feels like they could be the next Intel, but what do it know).

Dedicated chips for machine learning (inference) are being developed by many companies. The hope is that these will be used instead of (or in addition to) GPUs for ML tasks. Not that Nvidia is poorly positioned. In fact, I expect that if dedicated ML chips work out, Nvidia will also put one on the market.

> Nvidia will also put one on the market

Already done. Tegra Xavier includes DLA (deep learning accelerator).

Re: Real-Time Noise Suppression Using Deep Learning

#50

What's wrong with using multiple microphones? A mic element costs about thirty cents, and the processing power required for noise cancellation already exists in the CPU of the mobile device. I think it's particularly interesting that Amazon has made microphe arrays particularly cheap, due to Alexa. MiniDSP offers a microphone array for under $100, which is an unheard of price considering what these cost ten years ago…

How does multi microphone filtering work? I guess they localize different sound sources by cross-correlation (to get the timings) and triangulation (based on the timings and the speed of sound)?

You can do this with ICA (Independent component analysis, a somewhat lesser known, non-Gaussian cousin of principal component analysis). Basically you take the data with multiple components and break it down into its consistent component parts.
Post reply on HN