Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Android’s 10 Millisecond Problem explained

superpowered.com

Post reply on HN
Print view View original

171–180 of 311 posts

Re: Android’s 10 Millisecond Problem explained

#171

Post by jblow »

jblow wrote: post #152
cnvogel wrote: post #147

Earlier quoted context omitted.

Well... it's like all audio chipsets work nowadays. Only some DSPs will be able to efficiently handle single-frame data processing, but they have the help of dedicated address generators and lightweight interrupts synchronized to the digital interface. If you write "process however much audio data is ready", then you already imply that your CPU will not be up to speed to process 48000 interrupts/second reliably and y…

As someone who has actually done a good amount of soft-real-time audio programming, I can tell that you probably haven't. Everything you are saying about CPU speeds is made-up nonsense. Look into how these things are done on systems where folks actually care about latency (for example, commercial audio hardware, game consoles, etc).

I understand that people are downvoting this because it is just a negative comment, or something. But, I felt it was VERY important to call out information that is clearly false. Someone who doesn't know about audio programming might read the above post and think "hey that sounds plausible, I learned something today" when in fact they were deeply misled. Registering dissent is important and I tried not to be rude about it. I did go on to give a sketch of reasons in the thread below (but it is a complex issue with a lot of details; exact situations differ on every platform; etc, etc.)

Re: Android’s 10 Millisecond Problem explained

#172

Post by aidenn0 »

BFay wrote: post #125
digi_owl wrote: post #76

Earlier quoted context omitted.

>I remember in the days of kernel 2.2 that even XMMS would stop playing any music if I started using more than one or two applications. Back then the sound subsystem didn't do any mixing or similar, so if some program grabbed /dev/snd, everyone else had to wait. As for low latency sound work on Linux today, Jack is what you want rather than pulseaudio. Frankly Pulseaudio is a massive detour when it comes to Linux aud…

Pulseaudio has gotten a really bad rep, and I think there was at time where it was legitimately awful, but I think it's better than it was. On mainstream distros with pulseaudio like Fedora or Ubuntu, audio just works, when you don't have low-latency requirements. When you do have low-latency requirements, things are a bit tricky. You do pretty much need to get a low-latency or realtime kernel, and you definitely wan…

Ubuntu is the reason PA got a bad rap, and PA is an example of why I don't use Ubuntu.

When Ubuntu adopted PA, the readme file still described it as "the sound server that breaks your audio"

It was the most mature thing that had the features that Ubuntu wanted, so they adopted it despite the fact that it was clearly not yet ready for prime-time.

That all being said, PA is not the choice if you want to do DAW style stuff; it tends to prefer lower cpu utilization to lower-latency.

Re: Android’s 10 Millisecond Problem explained

#173

Post by cnvogel »

jblow wrote: post #171
jblow wrote: post #152

Earlier quoted context omitted.

As someone who has actually done a good amount of soft-real-time audio programming, I can tell that you probably haven't. Everything you are saying about CPU speeds is made-up nonsense. Look into how these things are done on systems where folks actually care about latency (for example, commercial audio hardware, game consoles, etc).

I understand that people are downvoting this because it is just a negative comment, or something. But, I felt it was VERY important to call out information that is clearly false. Someone who doesn't know about audio programming might read the above post and think "hey that sounds plausible, I learned something today" when in fact they were deeply misled. Registering dissent is important and I tried not to be rude abo…

I didn't downvote you and was genuinely interested in why you were considering my information to be incorrect. And I now realize it's because I've always worked with systems where processing is always strongly synced to the central frame/sample/... clock. Also I read your initial comment as "why don't we use 'process every single sample' to reduce latency at all costs" which is -as you wrote- clearly a bad idea. Sorry for misrepresenting that.

Re: Android’s 10 Millisecond Problem explained

#174

Post by nitrogen »

jblow wrote: post #165
cnvogel wrote: post #153

Earlier quoted context omitted.

Please tell me what claim I made about CPU speeds is unsubstantiated nonsense?

I could type up a thorough explanation, but it would take about an hour, and I have a lot to do. It is actually not a bad idea to do such a write-up, but I don't think the appropriate venue for it is an ephemeral post on Hacker News ... I'd rather blog it somewhere that's more suitable for long-term reference. But I'll drop a few hints. First of all, nobody is talking about running interrupts at 48kHz. That is comple…

The other commenter was talking about audio software that both consumes and produces samples at a fixed rate. Clearly, if the audio software is late grabbing 64 samples from the input device, it's also late delivering the next 64 to the output device, and there will be a dropout. The output sample clock has to be the timing master, and the software can never be late, and since it's also waiting for the input audio, it can never be early enough to "get ahead", either.

Re: Android’s 10 Millisecond Problem explained

#175

Post by jongraehl »

Compare to a nice grand piano: 1 meter of air ~3ms. (hammer travel after you bottom out the key - another 10ms? A MIDI keyboard doesn't know the velocity of the note (or send the note at all) until key travel is mostly done too.)

I have only 5 ms of ASIO buffering using my PC but I don't actually know how much actual key->sound latency; I do know that using headphones it's only slightly less immediate than a nice grand.

I think the low MIDI bit rate (31kbaud) also adds a little latency on chords.

It would be nice if keyboards in the future immediately sent a lower latency+precision keypress-initiated notice (before full key travel) so disk-based samplers can make ready for that note (and make any initial attack sound that's appropriate).

Re: Android’s 10 Millisecond Problem explained

#176

Post by tim333 »

kllrnohj wrote: post #149

Google has a whole section on this here (if you want more technicals and less sales-pitch): http://source.android.com/devices/audio/latency.html including a bunch of measurements of Nexus devices here (even going as far back as the Nexus One on Gingerbread): http://source.android.com/devices/audio/latency_measurements...

And here are some measurements for iOS as well (Superpowered Mobile Audio Latency Test App for Android and iOS.)

The iOS devices come out at 6-18ms, Androids at 17-860ms. The faster Androids have Samsung's Professional Audio SDK.

http://superpowered.com/latency/

Re: Android’s 10 Millisecond Problem explained

#177

Post by FigBug »

jblow wrote: post #165
cnvogel wrote: post #153

Earlier quoted context omitted.

Please tell me what claim I made about CPU speeds is unsubstantiated nonsense?

I could type up a thorough explanation, but it would take about an hour, and I have a lot to do. It is actually not a bad idea to do such a write-up, but I don't think the appropriate venue for it is an ephemeral post on Hacker News ... I'd rather blog it somewhere that's more suitable for long-term reference. But I'll drop a few hints. First of all, nobody is talking about running interrupts at 48kHz. That is comple…

> Since the one coming from the hardware is completely fictitious

Why do you say this? The USB audio card (or similar) is generating blocks of audio at a fixed rate, no?

Maybe for video playback or games you need to synchronize audio and video, but there is no need to do that for music production apps.

If you are writing some sort of synth, as soon as you receive a midi note or a tap, trigger the synth and the note will play in the next audio block. No need to wait for the GUI to update.

If you are doing some sort of effect, grab the input data, process and have it ready for the next block out. I don't understand why you need a second loop.

Re: Android’s 10 Millisecond Problem explained

#178

Post by weaksauce »

kabdib wrote: post #120

I'd heard over the years that working with isochronous systems was difficult. I'd done a number of real-time systems before, and written OS schedulers and NTP-like systems and so forth. A little audio work should be a walk in the park, right? A little manly-man programming from the wrist and we move on to real problems. So I walked into an audio project thinking that "Oh, this latency and synchronization stuff, how b…

A real postmortem of this would be a fascinating read.

Re: Android’s 10 Millisecond Problem explained

#179

Post by jblow »

nitrogen wrote: post #174
jblow wrote: post #165

Earlier quoted context omitted.

I could type up a thorough explanation, but it would take about an hour, and I have a lot to do. It is actually not a bad idea to do such a write-up, but I don't think the appropriate venue for it is an ephemeral post on Hacker News ... I'd rather blog it somewhere that's more suitable for long-term reference. But I'll drop a few hints. First of all, nobody is talking about running interrupts at 48kHz. That is comple…

The other commenter was talking about audio software that both consumes and produces samples at a fixed rate. Clearly, if the audio software is late grabbing 64 samples from the input device, it's also late delivering the next 64 to the output device, and there will be a dropout. The output sample clock has to be the timing master, and the software can never be late, and since it's also waiting for the input audio, i…

I am not sure we can make the assumption that the input and output devices are on the same clocks or run at the same rates. Maybe they are (in a good system you'd hope they would be), but I can think of a lot of cases where that wouldn't be true.

However, even when they are synced, you can still easily see the problem. The software is never going to be able to do its job in zero time, so we always take a delay of at least one buffer-size in the software. If the software is good and amazing (and does not use a garbage collector, for example) we will take only one delay between input and output. So our latency is directly proportional to the buffer size: smaller buffer, less latency. (That delay is actually at least 3x the duration represented by the buffer size, because you have to fill the input buffer, take your 1-buffer's-worth-of-time delay in the software, then fill the output buffer).

So in this specific case you might tend toward an architecture where samples get pushed to the software and the software just acts as an event handler for the samples. That's fine, except if the software also needs to do graphics or complex simulation, that event-handler model falls apart really quickly and it is just better to do it the other way. (If you are not doing complex simulation, maybe your audio happens in one thread and the main program that is doing rendering, etc just pokes occasional control values into that thread as the user presses keys. If you are doing complex simulation like a game, VR, etc, then whatever is producing your audio has to have a much more thorough conversation with the state held by the main thread.)

If you want to tend toward a buffered-chunk-of-samples-architecture, for some particular problem set that may make sense, but it also becomes obvious that you want that size to be very small. Not, for example, 480 samples. (A 10-millisecond buffer in the case discussed above implies at least a 30-millisecond latency).

Re: Android’s 10 Millisecond Problem explained

#180

Post by cnvogel »

nitrogen wrote: post #174
jblow wrote: post #165

Earlier quoted context omitted.

I could type up a thorough explanation, but it would take about an hour, and I have a lot to do. It is actually not a bad idea to do such a write-up, but I don't think the appropriate venue for it is an ephemeral post on Hacker News ... I'd rather blog it somewhere that's more suitable for long-term reference. But I'll drop a few hints. First of all, nobody is talking about running interrupts at 48kHz. That is comple…

The other commenter was talking about audio software that both consumes and produces samples at a fixed rate. Clearly, if the audio software is late grabbing 64 samples from the input device, it's also late delivering the next 64 to the output device, and there will be a dropout. The output sample clock has to be the timing master, and the software can never be late, and since it's also waiting for the input audio, i…

If everything happens strictly synchronous to your audio clock, then fixed block processing is the way to go.

But jblow is right in that when you have to feed in samples from a non-synchronized source into your processing/game/video-application/... then trying to work with the fixed audio block size will be terrible/require additional synchronization somewhere else, such as a adaptive resampler on the input/output of your "main loop".

Post reply on HN