Creating ad hoc microphone arrays from personal devices (2019)

Gets down to matching a single sound and working out the timing of that sound from the multiple sources. Then you also need to factor in the frequency response as well.

That last part would be important to handle things like the table the devices are sat upon picking up vibrations from the desk. Remember that phones don't have a rubber base to isolate them from the table so any vibration of that surface would propagate into the device and microphone. Then the whole aspect of varying devices and with that, varying microphone quality and device housings. So calibrating at some level would be key for this to work, though doable and processing wise you could even run a master device and handle the processing there and remove the server aspect with some of the processing done upon each local device and passed onto the main device for correlating. Certainly some phones have the power to handle this type of affair to replace the server aspect. But that would be more work/effort and something that may well see later on. Though makes it harder to sell a bit of server processing software then.

Though one test I'd like to see this system handle would be how well it filters out those vibrations.

After all you don't want to hear somebody writing or putting a cup or other object down whilst somebody else is talking.

I'd also wonder what type of jitter tolerances they are working with across those devices and how that scales with devices/jitter - does jitter increase after so many devices.

If we have beautiful, well-lit video feeds if every participant, but no one can hear what they're saying -- that's a deal breaker. The other way around, if we have clean, crisp audio from everyone and inconsistent video, at least the conversion can still move forward.

However the initial greeting at start of the meeting would be good enough to cover that. Though some feedback and constant recalibration would be ideal and doable, That covers things like people entering the room and briefly changing the rooms acoustics with the door open briefly. Then somebody closes a blind and things like that, even somebody moving a coffee cup on the table would have (whilst small) an impact upon the acoustics. Though in that last instance, somebody moving a cap nearer a device would have a bigger impact upon that single source.

Though easiest way would be having a sound source on the main camera that did a simple frequency sweep - if you wanted to use a reference point sound source for calibration. You may even get away with single calibration then, though dynamic calibration and using the meeting itself to constantly recalibrate, whilst more effort, would give a better result.

But be interesting seeing this in action and how they handle aspects like that.

Indeed, thinking it thru you could have each device as it joins into the meeting do a calibration tone sweep that the other devices would pick up. That approach may well be better as you could get a more accurate map of all microphones in relation to each other that way. So initial login/join of the devices would handle that aspect nicely.

The idea was that at a gig loads of people would record and you could reconstruct a much better recording.