Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An update on Android's audio latency

android-developers.googleblog.com

Post reply on HN
Print view View original

161–170 of 171 posts

Re: An update on Android's audio latency

#161

Post by donturner »

userbinator wrote: post #85

Up to now I've been referring to round-trip audio latency. Round-trip latency involves three components in the audio chain: audio input, audio processing and audio output. I read through the whole article and that's the most detail given about any testing methodology; I don't see any more description of how these numbers are being measured. My rule of thumb for computer speed measurements and efficiency has always be…

I'm the author. My testing methodology was to run the round-trip latency test using OboeTester: https://github.com/google/oboe/tree/master/apps/OboeTester/d....

It's a simple test: produce a tone (either on built-in speaker or over wired headphones if using a loopback dongle), measure the time it takes for that tone to reach the audio input.

For devices which I didn't have access to (our team has a limited number of test devices) I used the figures from Superpowered, but with some assumptions/rules:

#1 If AAudio was available on the device I used the measurements from that rather than OpenSL ES #2 No measurements from custom ROMs #3 I used the measurements from the latest version of Android which the OEM had released for that device. For example, if a device was originally released with Lollipop but it was possible to upgrade to Marshmallow then I used the figures for Marshmallow.

For touch screen latency measurements I used a WALT device (https://github.com/google/walt)

Re: An update on Android's audio latency

#162

Post by Grazester »

kuschku wrote: post #147
Grazester wrote: post #61

Earlier quoted context omitted.

Not true actually. Google now updates these devices for 3 years now and not just security updates

Tell that to my Pixel 1, still stuck on an ancient version without security updates.

That's an old device. Notice I said now(as of 2017). Google announced 3 year OS updates a year after the Pixel was released(which they specifically stated at launch would get 2 years of updates) The Pixel 2 got updates for 3 years. I know because I have one sitting next me right now.

Re: An update on Android's audio latency

#163

Post by tekstar »

kall wrote: post #158
tekstar wrote: post #87

Earlier quoted context omitted.

I released free AUv3 ports of mutable instruments modules, turned into instruments. Look up Spectrum Synthesizer Bundle on App Store.

These are excellent! And you also made synthmodes which I use all the time. Thank you!

Awesome! You are welcome!

Re: An update on Android's audio latency

#164

Post by chemag »

donturner wrote: post #161
userbinator wrote: post #85

Up to now I've been referring to round-trip audio latency. Round-trip latency involves three components in the audio chain: audio input, audio processing and audio output. I read through the whole article and that's the most detail given about any testing methodology; I don't see any more description of how these numbers are being measured. My rule of thumb for computer speed measurements and efficiency has always be…

I'm the author. My testing methodology was to run the round-trip latency test using OboeTester: https://github.com/google/oboe/tree/master/apps/OboeTester/d... . It's a simple test: produce a tone (either on built-in speaker or over wired headphones if using a loopback dongle), measure the time it takes for that tone to reach the audio input. For devices which I didn't have access to (our team has a limited number of…

I'm actually very interested in the details. I have a few questions, in case you have cycles to help me understand.

# audio latency

From your comment and the original "An update on Android's audio latency" article, there seems to be 2 different ways to calculate "audio latency".

* 1. play a well-known sound (a tone), listen for it in real-time

IIUC, the exact operation of this would be something like this:

  open recorder
  loop:
    timestamp1
* 2. measure audio loopback feedback (larsen effect)

Operation should be as follows:

  open recorder
  play audio file
  loop:
    keep listening to input frames for 1-2 seconds
  # analyze captured file
  look for 2x consecutive appearances of the original audio file
  sample := timestamp2 - timestamp1
Q1: Did I get this right? Any hunch about the pros&cons of both approaches?

# downlink/uplink latency

Many of the latency measurements break the latency down between audio downlink and uplink.

Q2: How is this measured?

The android doc discusses using a GPIO to have a zero-latency signal. You swap the signal on a well-known GPIO, and then play a well-known audio file. You then connect the GPIO to a speaker, and measure the latency between the GPIO-fed speaker, and the actual device speaker. As an optimization, you plug both the GPIO and the audio jack that feeds the actual device speaker into an oscilloscope, and get the distance there. Is this how audio downlink latency is measured?

Q3: If this is how downlink latency is measured, how do you implement it in prod devices (no access to the board)? Is this something that can be simulated using the usb-c plug?

# tools

The android doc/some googling points to several tools. I found oboetester, splatency, drrickorang, and google walt. I tested all but the walt one.

The first interesting thing is that oboetest, drrickorang, and walt seem to need an external jack-based dongle.

Q3: Why is this needed for? The audio latency approaches mentioned above should only need a speaker and a mic.

The experience has not been great for any of the tools. Operationally, I'd like to script these tests. Instead, I need to click on GUIs. Also, all the tools I tested seem to fail often, so I have to repeat the experiments multiple times. Also, I miss better documentation on exactly what they do.

Thanks for any help

edit: s/markdown/formatdoc/g

Re: An update on Android's audio latency

#165

Post by ladyanita22 »

fomine3 wrote: post #155
thefourthchime wrote: post #65

Earlier quoted context omitted.

What's funny is that this “open” environment of android now is nearly the opposite. With any android device openly spying on you while locking down all the open source apps you might want to install.

It never became worse than iOS on "open" perspective.

Yes, I'm all against data collection and I'm pro open source, but let's be honest, Android, despite its flaws (of which it has many), has always been the best MAJOR choice in that regard.

Of course you can always pick up a Librem 5 or a Pinephone, but I'd argue it comes with a lot of drawbacks.

Re: An update on Android's audio latency

#166

Post by kuschku »

Grazester wrote: post #162
kuschku wrote: post #147

Earlier quoted context omitted.

Tell that to my Pixel 1, still stuck on an ancient version without security updates.

That's an old device. Notice I said now(as of 2017). Google announced 3 year OS updates a year after the Pixel was released(which they specifically stated at launch would get 2 years of updates) The Pixel 2 got updates for 3 years. I know because I have one sitting next me right now.

Honestly, I don’t care. I was forced to buy a Pixel 1 because the Nexus 5X didn’t get updates anymore, and by the time the Pixel 1 stopped getting updates, I just gave up.

I won't support Android 11 and later in any of my apps. If someone has an issue with that, they're free to send me a device getting the Android Preview releases for the version they want me to support. I treat Android just like iOS now: Want support? buy me a devkit.

In the past it was possible for me to just build AOSP master from source and test with that, but 3.0 was the first release breaking that assumption and after 5.0 the assumption was broken entirely. Now I just won't support it anymore.

Re: An update on Android's audio latency

#167

Post by donturner »

chemag wrote: post #164
donturner wrote: post #161

Earlier quoted context omitted.

I'm the author. My testing methodology was to run the round-trip latency test using OboeTester: https://github.com/google/oboe/tree/master/apps/OboeTester/d... . It's a simple test: produce a tone (either on built-in speaker or over wired headphones if using a loopback dongle), measure the time it takes for that tone to reach the audio input. For devices which I didn't have access to (our team has a limited number of…

I'm actually very interested in the details. I have a few questions, in case you have cycles to help me understand. # audio latency From your comment and the original "An update on Android's audio latency" article, there seems to be 2 different ways to calculate "audio latency". * 1. play a well-known sound (a tone), listen for it in real-time IIUC, the exact operation of this would be something like this: open recor…

Below is a collaborative answer from myself and my colleague, Phil Burk who is a SWE on the audio framework team.

Thanks for your interest in the details of Android Latency measurement techniques.

Some of the measurements were collected by third parties. We cannot describe their techniques but they are probably similar to ours.

We use OboeTester to measure latency. You can find a description of how to measure Tap-to-Tone Latency and Round Trip Latency in this doc: https://github.com/google/oboe/blob/master/apps/OboeTester/d...

We do not use the Larsen Effect any more because it was too sensitive to variations in gain. We now use a random encoded bit stream that sounds like a short noise burst. We can get a better correlation peak with that signal.

> Many of the latency measurements break the latency down between > audio downlink and uplink.

It is very hard to separate the input and output latency without special hardware (like the WALT device). You can measure combined input+output latency using a loopback test. Input latency tends to be much lower than output latency. This is because, when the full duplex stream is stable, the input buffer is close to empty and the output buffer is close to full. Then, if there is a preemption, the input buffer fills up and the output buffer drains, providing glitch protection.

You can measure touch+output latency using tap-to-tone. The screen touch latency is about 15-30 msec. If you use a hardware MIDI controller (like a keyboard or drum pad) instead of tapping the touch screen then you can get a “touch” latency of about 1 msec (MIDI is a lightweight and low latency protocol). Then you can get a better estimate of just the output latency.

> The android doc discusses using a GPIO to have a zero-latency signal.

We don’t normally use that technique because it requires special hardware.

> I found oboetester, splatency, drrickorang, and google walt.

Our group supports OboeTester.

> The first interesting thing is that oboetest, drrickorang, and walt seem to need an external jack-based dongle.

The lowest latency path is usually over the headphone jack (either through 3.5mm or USB dongle). To test this path you do need a "jack-based dongle" aka loopback adapter.

The reason why wired headphones usually give you lower latency is that OEMs often introduce digital signal processing for the speaker to improve the acoustics/quality, which can introduce additional latency. Side note: this is why you will often see "best with headphones" on games and music apps.

That said, OboeTester will work over the speakers and mic in a quiet room. That is because the new random bit technique is more robust.

> I'd like to script these tests.

OboeTester can be scripted. We use it for continuous integration testing. https://github.com/google/oboe/blob/master/apps/OboeTester/d...

Re: An update on Android's audio latency

#168

Post by chemag »

donturner wrote: post #167
chemag wrote: post #164

Earlier quoted context omitted.

I'm actually very interested in the details. I have a few questions, in case you have cycles to help me understand. # audio latency From your comment and the original "An update on Android's audio latency" article, there seems to be 2 different ways to calculate "audio latency". * 1. play a well-known sound (a tone), listen for it in real-time IIUC, the exact operation of this would be something like this: open recor…

Below is a collaborative answer from myself and my colleague, Phil Burk who is a SWE on the audio framework team. Thanks for your interest in the details of Android Latency measurement techniques. Some of the measurements were collected by third parties. We cannot describe their techniques but they are probably similar to ours. We use OboeTester to measure latency. You can find a description of how to measure Tap-to-…

Thanks Don and Phil for the great answer. Some further comments:

* I find interesting that you are getting better correlations when you add a well-known noise burst instead of a less chaotic signal (e.g. a tone or a chirp). In retrospect, it makes sense

* for the uplink/downlink breakup, I see in the Usage.md file in oboetester that you're isolating the downlink measurement with the tap-to-tone experiment. For this experiment, the doc suggests to use (a) the jack to avoid the speaker processing extra latency, and (b) a USB-MIDI input device to replace the touch screen latency (15-30 ms).

2 questions here:

* Q1: I assume that, if instead of the jack, you use a usb-c audio adapter accessory mode, there should be no extra latency either, right? (I'm using late pixel phones)

* Q2: which device are you using for the USB-MIDI input?

Thanks again!

edit: s/markdown/formatdoc/g

Re: An update on Android's audio latency

#169

Post by donturner »

chemag wrote: post #168
donturner wrote: post #167

Earlier quoted context omitted.

Below is a collaborative answer from myself and my colleague, Phil Burk who is a SWE on the audio framework team. Thanks for your interest in the details of Android Latency measurement techniques. Some of the measurements were collected by third parties. We cannot describe their techniques but they are probably similar to ours. We use OboeTester to measure latency. You can find a description of how to measure Tap-to-…

Thanks Don and Phil for the great answer. Some further comments: * I find interesting that you are getting better correlations when you add a well-known noise burst instead of a less chaotic signal (e.g. a tone or a chirp). In retrospect, it makes sense * for the uplink/downlink breakup, I see in the Usage.md file in oboetester that you're isolating the downlink measurement with the tap-to-tone experiment. For this e…

> * Q1: I assume that, if instead of the jack, you use a usb-c audio adapter accessory mode, there should be no extra latency either, right? (I'm using late pixel phones)

That's correct, by using either the 3.5mm jack, or USB-C adapter you won't incur any additional latency introduced by DSP to improve the speaker acoustics.

However, the USB path typically does have a few ms higher latency than the 3.5mm jack path.

> * Q2: which device are you using for the USB-MIDI input?

We've used a variety of devices and found the latency differences to be negligible. At the moment I test with an old AKAI LPK25.

Re: An update on Android's audio latency

#170

Post by cma »

thomasahle wrote: post #56
kall wrote: post #51

Earlier quoted context omitted.

This data [0] doesn't have the latest models, and it's meassuring latency internaly, not "touch-to-sound". The ones they tested average around 7-8ms. I've heard of Edit: I've been thinking about this and a touch-to-tone latency below 10ms seems imposible, even with the 120hz digitizer on newer iPhones. I would be very interested in seeing some experimental data as well. [0] https://juce.com/discover/stories/Mobile%20…

The best I could find was https://developer.apple.com/forums/thread/71301 which suggests a 32ms touch latency. I think that might include some processing, but it doesn't seem much better than the 20ms latency in Android mentioned in the article. The audio processing latency seems to be single digit on both systems. In any case, I don't think these differences are big enough to argue that "Android can't be used for in…

There are 120hz ipads, so wouldn't that drop to 16ms?

A physical piano with a 1.2" hammer blow distance played at soft velocity (~2.2mph), assuming distance of closest resonating surface to ear is half meter, would have around 35ms latency.

Post reply on HN