Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do browsers isolate internal audio from microphone input?

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 106 posts

Re: Ask HN: How do browsers isolate internal audio from microphone input?

#41

Post by duped »

robertlagrant wrote: post #39
duped wrote: post #19

Earlier quoted context omitted.

> Basically every application that uses microphone input will want to do this The OS doesn't have more information about this than applications and it's not that obvious whether an application wants the OS to fuck around with the audio input it sees. Even in the applications where this might be the obvious default behavior, you're wrong - since most listeners don't use loudspeakers at all, and this is not a problem w…

Assuming this isn't parody, the OS doesn't have to do it automatically. Having an application grab a microphone stream and say to the OS "take this and cancel any audio out streams" might be pretty useful.

I agree with that, but the point I'm trying to make is that audio i/o handling is pretty complicated and application specific. The idea I'm challenging is that "any app that wants microphone input wants this" is dubious. I'd say it's only a small number of audio applications that care about mic input want background noise reduced - and it makes sense for this to be configured per-input stream.

Really what would be nice is if every audio i/o backend supported multiplex i/o streams and you could configure whether or not to cancel audio based on that set of streams but not all output (because multi output-device audio gets tricky).

Re: Ask HN: How do browsers isolate internal audio from microphone input?

#42

Post by asveikau »

gpvos wrote: post #13
padenot wrote: post #7

The way this works (and I'm obviously taking a high level view here) is by comparing what is being played to what is being captured. There is an inherent latency in between what is called the capture stream (the mic) and the reverse stream (what is being output to the speakers, be it people taking or music or whatever), and by finding this latency and comparing, one can cancel the music from the speech captured. With…

It just seems more logical for the OS to do that, rather than the application. Basically every application that uses microphone input will want to do this, and will want to compensate for all audio output of the device, not just its own. Why does the OS not provide a way to do this?

The OS doesn't know that the application doesn't want feedback from the speaker, and not 100% of applications will want such filtering. I think a best practice from the OS side would be to provide it as an optional flag. (Default could be on or off, with reasonable possibility for debate in either direction, but an app that really knows what it wants should be able to ask for it.)

Re: Ask HN: How do browsers isolate internal audio from microphone input?

#43

Post by ygjb »

Wowfunhappy wrote: post #23
meindnoch wrote: post #18

Search for the compilation flag "CHROME_WIDE_ECHO_CANCELLATION" in the Chromium sources, and you will find your answer. Can't tell you anything else due to NDAs.

It's kind of nuts that (I'm assuming) the source code is publicly available but the developers who wrote it can't talk about it. (I realize this situation isn't up to you and I appreciate that you chimed in as you could!)

This is super common.

When I worked at Mozilla, most stuff was open, but I still couldn't talk about stuff publicly because I wasn't a spokesperson for Mozilla. Same at OpenDNS/Cisco, or at Fastly, and now at Amazon. Lots of stuff I can talk about, but I generally avoid threads and comments about Amazon, or if I do, it's strictly to reference public documentation, public releases, or that sort of thing.

It's easier to simply not participate, link a document, or say no comment than it is to cross reference what I might say against what's public, and what's not.

Re: Ask HN: How do browsers isolate internal audio from microphone input?

#44

Post by swatcoder »

gpvos wrote: post #13
padenot wrote: post #7

The way this works (and I'm obviously taking a high level view here) is by comparing what is being played to what is being captured. There is an inherent latency in between what is called the capture stream (the mic) and the reverse stream (what is being output to the speakers, be it people taking or music or whatever), and by finding this latency and comparing, one can cancel the music from the speech captured. With…

It just seems more logical for the OS to do that, rather than the application. Basically every application that uses microphone input will want to do this, and will want to compensate for all audio output of the device, not just its own. Why does the OS not provide a way to do this?

> Why does the OS not provide a way to do this?

Some do.

But you need to have a strong-handed OS team that's willing to push everybody towards their most modern and highly integrated interfaces and sunset their older interfaces.

Not everybody wants that in their OS. Some want operating systems that can be pieced together from myriad components maintained by radically different teams, some want to see their API's/interfaces preserved for decades of backwards compatibility, some want minimal features from their OS and maximum raw flexibility in user space, etc

Re: Ask HN: How do browsers isolate internal audio from microphone input?

#45

Post by dfc »

swatcoder wrote: post #44
gpvos wrote: post #13

Earlier quoted context omitted.

It just seems more logical for the OS to do that, rather than the application. Basically every application that uses microphone input will want to do this, and will want to compensate for all audio output of the device, not just its own. Why does the OS not provide a way to do this?

> Why does the OS not provide a way to do this? Some do. But you need to have a strong-handed OS team that's willing to push everybody towards their most modern and highly integrated interfaces and sunset their older interfaces. Not everybody wants that in their OS. Some want operating systems that can be pieced together from myriad components maintained by radically different teams, some want to see their API's/inte…

> Some do

Which Operating systems do this?

Re: Ask HN: How do browsers isolate internal audio from microphone input?

#46

Post by kbolino »

gpvos wrote: post #13
padenot wrote: post #7

The way this works (and I'm obviously taking a high level view here) is by comparing what is being played to what is being captured. There is an inherent latency in between what is called the capture stream (the mic) and the reverse stream (what is being output to the speakers, be it people taking or music or whatever), and by finding this latency and comparing, one can cancel the music from the speech captured. With…

It just seems more logical for the OS to do that, rather than the application. Basically every application that uses microphone input will want to do this, and will want to compensate for all audio output of the device, not just its own. Why does the OS not provide a way to do this?

This assumes there is an OS-managed software mixer sitting in the middle of all audio streams between programs and devices. Historically, that wasn't the case, because it would introduce a lot of latency and jitter in the audio. I believe it is still possible for a program to get exclusive access to an audio output device on Windows (WASAPI) and Linux (ALSA).

Re: Ask HN: How do browsers isolate internal audio from microphone input?

#47

Post by Wowfunhappy »

duped wrote: post #41
robertlagrant wrote: post #39

Earlier quoted context omitted.

Assuming this isn't parody, the OS doesn't have to do it automatically. Having an application grab a microphone stream and say to the OS "take this and cancel any audio out streams" might be pretty useful.

I agree with that, but the point I'm trying to make is that audio i/o handling is pretty complicated and application specific. The idea I'm challenging is that "any app that wants microphone input wants this" is dubious. I'd say it's only a small number of audio applications that care about mic input want background noise reduced - and it makes sense for this to be configured per-input stream. Really what would be ni…

I'm honestly having trouble thinking of a case where I wouldn't want this.

I'm sure there are some niche cases, but in those cases, the application can specifically request that the OS turn off audio isolation.

Re: Ask HN: How do browsers isolate internal audio from microphone input?

#48

Post by dolni »

duped wrote: post #40
fsckboy wrote: post #37

Earlier quoted context omitted.

> The OS doesn't have more information about this than applications the OP pointed out that this only works if he uses a browser monoculture the OS does have more information than that, it can know what is being played by any/all apps, and what is being picked up by the mic

The "OS" isn't special here, apps can listen to system audio. fwiw, you only need to know anything about outputs if you are doing AEC. Blind source separation doesn't have that problem and can just process the input stream.

> The "OS" isn't special here, apps can listen to system audio.

Even if this is true, it's easy to imagine such functionality being exploited by malicious apps as a security and/or privacy concern, particularly if the user needs a screen reader.

It definitely makes sense for the operating system to provide this functionality.

Re: Ask HN: How do browsers isolate internal audio from microphone input?

#49

Post by mananaysiempre »

supriyo-biswas wrote: post #14

The technical term that you're looking for is acoustic echo cancellation[1]. It's a fairly common problem in signal processing, and comes up in "simple" devices like telephones too. [1] https://www.mathworks.com/help/audio/ug/acoustic-echo-cancel...

I seem to remember analog telephone lines used a very simple but magic-looking transformer-based circuit of some sort for this purpose. Presumably that worked because they didn’t need to worry about a processing delay?

Re: Ask HN: How do browsers isolate internal audio from microphone input?

#50

Post by swatcoder »

Wowfunhappy wrote: post #47
duped wrote: post #41

Earlier quoted context omitted.

I agree with that, but the point I'm trying to make is that audio i/o handling is pretty complicated and application specific. The idea I'm challenging is that "any app that wants microphone input wants this" is dubious. I'd say it's only a small number of audio applications that care about mic input want background noise reduced - and it makes sense for this to be configured per-input stream. Really what would be ni…

I'm honestly having trouble thinking of a case where I wouldn't want this. I'm sure there are some niche cases, but in those cases, the application can specifically request that the OS turn off audio isolation.

The technique introduces latency and distortion because it's subtracting an estimate of sound that's traveling/reflecting in the listening environment, which is imperfect and involves the speed of sound.

That latency is within the tolerance that users are comfortable with for voice chat, and much less than video processing/transfer is introducing for video calls anyway, so it's a very obvious win there. Especially since those users are most interested in just picking out clear words using whatever random mic/speaker configuration happens to be most convenient.

But musicians, for instance, are much more interested in minimizing the delay between their voice or instrument being captured and returned through a monitor, and they generally choose a hardware arrangement that avoids the problem in the first place. And that's not really a niche use case.

Post reply on HN