Live data from Hacker News

Creating ad hoc microphone arrays from personal devices (2019)

microsoft.com

41–50 of 55 posts

Re: Creating ad hoc microphone arrays from personal devices (2019)

#41

This is a really interesting technical concept. Capturing high-quality audio in a meeting room for videoconferencing is a notoriously complicated problem. Microphones are crazy sensitive and pick up things like footsteps and conversations outside the door, shuffling feet and tapping on keyboards, and construction and HVAC noise like you wouldn't believe. So filtering those things out, and then capturing the best qual…

"Please, before we start the meeting, can everyone in the room allow app microphone access for the best experience?"

Re: Creating ad hoc microphone arrays from personal devices (2019)

#42

There are obvious(?) privacy issues and what not here. But ignoring all that for a second, it does sound pretty cool to be able to leverage all the little computers we walk around with. Think of all those shitty little video clips people take at a concert. Could all those be combined to make some high quality panoramic video? Probably a lot of other cool applications that I can't even comprehend for now. What a time…

Looks like they're going to leverage only our microphones, not the computers. Just another plausible way to suck even more data into the cloud.

"Going forward, we will fund business ideas that: allow microphone access, allow camera access, allow location services, allow calendar access, allow..."

It's like webex - it turns on 24x7 microphone access "to detect nearby video devices"

Re: Creating ad hoc microphone arrays from personal devices (2019)

#43
post #22

This is a really interesting technical concept. Capturing high-quality audio in a meeting room for videoconferencing is a notoriously complicated problem. Microphones are crazy sensitive and pick up things like footsteps and conversations outside the door, shuffling feet and tapping on keyboards, and construction and HVAC noise like you wouldn't believe. So filtering those things out, and then capturing the best qual…

It's a hard and interesting signals problem with surely many other benefits but surely money would be better spent just buying better mics and audio gear for an office.

Or better gear that uses it.

https://www.shure.com/en-US/products/microphones/mxa910

Re: Creating ad hoc microphone arrays from personal devices (2019)

#44
post #18

Earlier quoted context omitted.

That doesn't work for multiple people in the same room all wearing AirPods. Everyone's mic picks up everyone's voice, not just the "real" speaker. And a lot of meetings have most (e.g. 10) people in a single room, with another handful (e.g. 5) of remote participants.

How about a regular headset?

Most headsets come with omnidirectional microphones. Even some "noise cancelling" microphones are actually omnidirectional, just with an arm long enough to be reasonably close to the mouth. When I recently decided I need a cardioid microphone on my headset, I've ended up spending about $300 total.

Re: Creating ad hoc microphone arrays from personal devices (2019)

#45
post #28

This is a really interesting technical concept. Capturing high-quality audio in a meeting room for videoconferencing is a notoriously complicated problem. Microphones are crazy sensitive and pick up things like footsteps and conversations outside the door, shuffling feet and tapping on keyboards, and construction and HVAC noise like you wouldn't believe. So filtering those things out, and then capturing the best qual…

> And it's well-known that the "bowling alley" visual of meeting participants (camera at the end of a long conference table) isn't ideal. If each participant has their own laptop camera on themselves, it could be a vastly better experience for remote participants. My company pushes us to have any conference that will include remote people from our desks, even if some or most of the attendees are in the same physical…

> have any conference that will include remote people from our desks,

This is great for the participants, but absolute hell for everyone else in open offices, or even shared offices.

Re: Creating ad hoc microphone arrays from personal devices (2019)

#46
Excerpt:

"While the idea sounds simple, it requires overcoming many technical challenges to be effective. The audio quality of devices varies significantly. The speech signals captured by different microphones are not aligned with each other. The number of devices and their relative positions are unknown. For these reasons and others, consolidating the information streams from multiple independent devices in a coherent way is much more complicated than it may seem. In fact, although the concept of ad hoc microphone arrays dates back to the beginning of this century, to our knowledge it has not been realized as a product or public prototype so far."

Thoughts:

There's something deep here, not with respect to microphones and speech transcription (although I wish Microsoft and whoever else attempts to wrestle with those problems the greatest of success!)

There's a related deep problem in physics here.

If we consider signals that emanate from outer space, let's say they're from the big bang, or heck, let's just say they're from one of our past-the-edge-of-this-solar-system satelites -- that wants to communicate back to earth.

Well, due to the incredible distances involved, the signal will get garbled in various ways...

So here's the $64,000 question:

When that signal from deep space gets garbled, isn't it possible that it turns into various other signals, at various different other frequencies and wavelengths?

In other words, space itself, over long distances, acts as a prism (not really, but as an easy way to wrap your mind around this concept), for radio, and other electromagnetic waves...

Now, if you want to reconstruct the orignal message at these long distances, you must be able to reconstruct garbled radio (and other em) waves, which are moving at different frequencies, and may even arrive at the destination at different rates of speed with various time shifts...

Basically, you've got to take those pieces -- move them to the correct frequency, time correct them, speed them up or slow them down, sync them, and overlay them -- to reconstruct the original message...

That's the greater question in physics -- the ability to do all of that, with em signals from a long way off in space...

The article referenced -- is the microphone/audio/slow speed equivalent -- of that larger problem...

Re: Creating ad hoc microphone arrays from personal devices (2019)

#47

Earlier quoted context omitted.

> This could actually be a huge step forward in not needing videoconferencing equipment in meeting rooms. So far, one of the biggest reasons has actually been dealing with echo and feedback -- when people are in the same call with multiple devices in the same room, it tends to end badly. But if the audio processing is designed for that... the results could actually be quite amazing. > And it's well-known that the "bo…

That doesn't work for multiple people in the same room all wearing AirPods. Everyone's mic picks up everyone's voice, not just the "real" speaker. And a lot of meetings have most (e.g. 10) people in a single room, with another handful (e.g. 5) of remote participants.

Airpods don't really capture other people's voices really well. It's basically inaudible unless you've got it plugged in your ear pointed at your mouth

Re: Creating ad hoc microphone arrays from personal devices (2019)

#48
post #26

Does anyone have any insight into why neural nets are used for the "blind" beamforming? I don't have first hand experience with machine learning, but this just doesn't seem to me like a machine learning type of problem. I get it's not trivial, but it seems like there should be an analytic solution - more or less

Acoustics are modified in extremely non-linear ways depending on the shape of the room, bodies within it, materials, acoustic reflection, acting differently at different frequencies, and so on.

In theory if the entire 3D layout and material properties were known known in advance you could get clear audio analytically. But reverse-engineering the 3D layout and materials from existing audio is essentially impossible.

So machine learning is used to find approximate solutions that work.

Re: Creating ad hoc microphone arrays from personal devices (2019)

#49
post #20

Earlier quoted context omitted.

> Capturing high-quality audio in a meeting room for videoconferencing is a notoriously complicated problem Not from my experience of 20 years ago setting up VC systems, biggest issue was video and making sure lighting was good, and plane wall behind (sky blue was good colour for that). Audio wise, was many desk standing mic's (can't recall main brand) but was a few. Did have one issue once with setting up a connecti…

Quality video is definitely hard too, but it's just not as important. If we have beautiful, well-lit video feeds if every participant, but no one can hear what they're saying -- that's a deal breaker. The other way around, if we have clean, crisp audio from everyone and inconsistent video, at least the conversion can still move forward.

Even just watching a random clip on Youtube, it's fairly easy to forgive a low quality video feed but bad audio gets really annoying very quickly. Any lag or stuttering or artifacts etc in the audio is a dealbreaker for most people.

Re: Creating ad hoc microphone arrays from personal devices (2019)

#50
post #44
post #18

Earlier quoted context omitted.

How about a regular headset?

Most headsets come with omnidirectional microphones. Even some "noise cancelling" microphones are actually omnidirectional, just with an arm long enough to be reasonably close to the mouth. When I recently decided I need a cardioid microphone on my headset, I've ended up spending about $300 total.

Which headset was that? I’ve been looking for one with a cardioid mic but wasn’t able to track any down.
Post reply on HN