Live data from Hacker News

Google Meet rolls out multi-device adaptive audio merging

workspaceupdates.googleblog.com

271–279 of 279 posts

Re: Google Meet rolls out multi-device adaptive audio merging

#271

Earlier quoted context omitted.

Do you think they are using some kind of inaudible-to-humans signal to coordinate this or is the unmodified audio good enough?

I could see this happening. Zoom for example already uses ultrasonic sound to detect what conference room a laptop is in for one-click screen sharing.

Not great if you have dogs in the office unfortunately.

Re: Google Meet rolls out multi-device adaptive audio merging

#272
post #261

Earlier quoted context omitted.

But weren't old remotes all IR?

No, not all were IR. But all of them made by "LG" and featuring a single-chip design are either IR or RF.

Those will still have a ceramic resonator in them, as they’re cheaper than a quartz crystal. Hard to imagine them being audible, but not impossible.

Re: Google Meet rolls out multi-device adaptive audio merging

#273
post #260
post #242

Earlier quoted context omitted.

How would you do this on the os? I would honestly try to sync audio output based on a shared time reference, something along the lines of what AES67/Ravenna/Dante does but you can be a little more lax and use ntp or system time since you don't need to be sample accurate. For the microphones that would be a little harder but you should be aware it's not that tough since a few high end manufacturers have phased microph…

My money is that outside of a couple of dog and pony demos with everyone on one well-administered LAN you could not make this work with system time and NTP on consumer devices. You will regularly see 100ms difference in NTP time. The fact that phased array microphones exist has nothing to do with the point we are discussing, which is audio coherence across heterogenous devices whose only real connection is a web brow…

You don't need the time to actually be synced.

I'm thinking more some sort of system with a sync point registered per device and using that as a time reference.

It's not inconceivable that they could easily detect multiple devices in a room and find a sync point based on microphone input from a speaker.

Once you have a sync point found you can then set a delay on all devices to try to match that sync point. Nobody said this is easy or everyone would be doing it but it's simple enough.

The phased array microphones is more a pipe dream but you wpuld absolutely be able to do something approaching that with multiple devices on a single room depending on how accurately you can predict microphone location within the room. Im reasonably sure you could start by just using the closest mic and then over timr as you improve sync you can try to use multiple.

As I said they get every single audio stream in and out into their servers and they have full control of the audio the tab is playing and the timing of that.

I don't see this being any different to what the likes of Sonos/Google Home/ Apple Home etc are doing with synced appliances for stereo/ multichannel devices, it's likely significantly harder because it's heterogeneous devices as you said.

All that doesn't answer my question of how you would do this at the OS level? You don't have any of the required information per device, only the central server has even the hope of having all the relevant information and control.

Re: Google Meet rolls out multi-device adaptive audio merging

#274

I just want to say that this is really actually kind of mind-blowing from an audio engineering perspective. Outputting audio from multiple laptops in the same room is easy. Perfectly syncing it is harder. Implementing echo cancellation across all of that is quite a bit trickier than regular single-device echo cancellation. But then treating all the laptop microphones as a kind of microphone array, having to deal with…

I assume it's just some neural network doing the heavy lifting, or not?

Is a neural network even really necessary? This seems like something where careful application of some normal math would work. Find a loud event, correlate the loud event across devices to get offsets (by sequence ID, not clock time), do some fancy math to apply inverse sin waves or what not.

That's not to diminish the accomplishment or say that it's easy (or that I could do it), but I don't think a neural network is necessary here.

Re: Google Meet rolls out multi-device adaptive audio merging

#275

I'm surprised no one here has mentioned around.co yet. They've had this feature for years which was the main reason our team has been using it. Good to see other services catching up.

just came by to see if anybody mentioned around.co

super cool tech, but was hard to convince my org to use it over Zoom/Meet

Re: Google Meet rolls out multi-device adaptive audio merging

#276

Earlier quoted context omitted.

… no? No! That's terrible advice. The meeting room (should) have a much higher quality mic than an attendee's laptop. You want that mic. Worse, when attendees use their mic, someone in the room asks a question, which remote viewers cannot/do not hear, because it is not picked up by the mic on the laptop. Or they join and don't mute, resulting in feedback loops. I do feel the need for Google's feature though: the pand…

Each person would ask a question on their mic, they're not using one shared laptop

I have literally never worked in a place with coworkers who understand VC tech well enough to understand what you're suggesting and who are diligent enough to actually do that. Getting that level of cognizance, over the entire seated body of a conference room, for the entire meeting, is unicorn territory IME.

Re: Google Meet rolls out multi-device adaptive audio merging

#277
post #273
post #260

Earlier quoted context omitted.

My money is that outside of a couple of dog and pony demos with everyone on one well-administered LAN you could not make this work with system time and NTP on consumer devices. You will regularly see 100ms difference in NTP time. The fact that phased array microphones exist has nothing to do with the point we are discussing, which is audio coherence across heterogenous devices whose only real connection is a web brow…

You don't need the time to actually be synced. I'm thinking more some sort of system with a sync point registered per device and using that as a time reference. It's not inconceivable that they could easily detect multiple devices in a room and find a sync point based on microphone input from a speaker. Once you have a sync point found you can then set a delay on all devices to try to match that sync point. Nobody sa…

We agree that doing it at the OS level is probably the wrong direction. I think you could get there with PNTP and audio hardware support, which is more how Sonos etc do it afaik but then again you aren’t solving the heterogenous device problem.

It is apparently a good example of something that needs performant neural nets in the cloud to solve. At first glance it looks like a low-level hardware-firmware problem. Market conditions prevent solving it at that level though, so we had to wait for the right combination of resources, new signal processing and heavy cloud compute.

Re: Google Meet rolls out multi-device adaptive audio merging

#278

Earlier quoted context omitted.

Yeah I feel like with a couple UI tweaks or options Meet would be the best by far. It's just so much more convenient and accessible than the others.

Just in case any Meet engineers are reading... It needs to let me put my self-view right under the webcam lens, so I can stare at myself and still be looking at the camera.

FaceTime deals with this by changing your eyeballs realtime

Re: Google Meet rolls out multi-device adaptive audio merging

#279
post #233

Earlier quoted context omitted.

Meet requires a Google account to attend a video call while Zoom does not. So if your hosting a meeting with strangers, using Meet is not user friendly. At least personally I refuse to make a Google account bc it requires giving Google my phone number... But I know that's a bit of technoludditism in the current zeitgeist

You can join a Google Meet meeting without a Google account if the meeting creator enables the “Anyone with the link can join” setting, or if they accept you when you click the link. https://support.google.com/meet/answer/9303069?hl=en&co=GENI...

It's been over 6 months since I've looked into this, so it's possible my info is out of date - but the “Anyone with the link can join” setting is only available to people with a paid Google business account. Nobody in my department had anything like that
Post reply on HN