Live data from Hacker News

Android’s 10 Millisecond Problem explained

superpowered.com

161–170 of 311 posts

Re: Android’s 10 Millisecond Problem explained

#161

Earlier quoted context omitted.

Trying to make Pulse work is a big mistake. The first step on any Linux setup is ensuring that it isn't installed. One can get very short latency out of Alsa, up to the point where the hardware becomes your bottleneck. But that's extremely processor intensive, and won't work well if you try to share the dsp with several processes (if you want to get that extreme, I'd recommend you get extra hardware for exclusive use…

I am talking about my day-to-day workstation and laptop running (x)ubuntu. Pulse may not be great, but nowadays I get to use Skype, watch videos, and listen to music all day using Pulse... why should I go through all the trouble of removing it if it works for the most common tasks? About the interface. I doubt that was the problem. When I got jack to work, I was getting 2-3ms latency between input and processed outpu…

> why should I go through all the trouble of removing it if it works for the most common tasks?

Well, your question that implied you wanted it. Although, yes, the pedal is probably a better choice after all.

Anyway, Pulse is only needed for advanced tasks of streaming sound through a network, using application based mixing settings, etc. If you are only doing common tasks, they'll almost certainly keep working without it. It's one of those cases of an apt-get and you are done.

Yes, I went to Unicamp, 99's class. Is your nick based on your name?

Re: Android’s 10 Millisecond Problem explained

#162
post #104

There is something I don't understand; maybe someone here can explain: Sound travels at about 340 m/s (in a typical room). That means it travels about 3.4 metres in 10 milliseconds. Therefore another way to get a 10 millisecond problem is to stand 3.4 metres from the orchestra. Most people sit farther than 3.4 metres from the orchestra, yet they don't complain about a lag between when the violin bow moves and the sou…

You are exactly right. This is why orchestras need a conductor who provides a visual signal for tempo, and marching bands have a drum major with a huge baton, while rock bands can just listen to the bass drum.

Re: Android’s 10 Millisecond Problem explained

#164
Great explanation, thanks for taking the time to write this up. A while back I figured that there had to be complicated structural reasons for the lack of progress on the notorious "issue 3434", and I decided to go with native iOS for mobile audio projects rather than wait. Seems it will be quite a difficult problem to solve (although perhaps a library that is not totally backwards-compatible would be easier to optimize). But kudos for taking a crack at it, and good luck. I'm interested to hear how it goes.

Re: Android’s 10 Millisecond Problem explained

#165
post #152

Earlier quoted context omitted.

As someone who has actually done a good amount of soft-real-time audio programming, I can tell that you probably haven't. Everything you are saying about CPU speeds is made-up nonsense. Look into how these things are done on systems where folks actually care about latency (for example, commercial audio hardware, game consoles, etc).

Please tell me what claim I made about CPU speeds is unsubstantiated nonsense?

I could type up a thorough explanation, but it would take about an hour, and I have a lot to do. It is actually not a bad idea to do such a write-up, but I don't think the appropriate venue for it is an ephemeral post on Hacker News ... I'd rather blog it somewhere that's more suitable for long-term reference.

But I'll drop a few hints. First of all, nobody is talking about running interrupts at 48kHz. That is complete nonsense.

The central problem to solve is that you have two loops running and they need to be coordinated: the hardware is running in a loop generating samples, and the software is running in a (much more complicated) loop consuming samples. The question is how to coordinate the passing of data between these with minimal latency and maximum flexibility.

If you force things to fill fixed-size buffers before letting the software see them (say, 480 samples or whatever), then it is easy to see problems with latency and variance: simply look at a software loop with some ideal fixed frame time T and look at what happens when T is not 100Hz. (Let's say it is a hard 60Hz, such as on a current game console). See what happens in terms of latency and variance when the hardware is passing you packets every 10ms and you are asking for them every 16.7ms.

The key is to remove one of these fixed frequencies so that you don't have this problem. Since the one coming from the hardware is completely fictitious, that is the one to remove. Instead of pushing data to the software every 10ms, you let the software pull data at whatever rate it is ready to handle that data, thus giving you a system with only one coarse-grained component, which minimizes latency.

You are not running interrupts at 48kHz or ten billion terahertz, you are running them exactly when the application needs them, which in this case is 16.7ms (but might be 8.3ms or 10ms or a variable frame rate).

You don't have to recompute any of the filters in your front-end software based on changing amounts of data coming in from the driver. The very suggestion is nonsense; if you are doing that, it is a clear sign that your audio processing is terrible because there is a dependency between chunk size and output data. It should be obvious that your output should be a function of the input waveform only. To achieve this, you just save up old samples after you have played them, and run your filter over those plus the new samples. None of this has anything to do with what comes in from the driver when and how big.

Edit: I should point out, by the way, that this extends to purely software-interface issues. Any audio issue where the paradigm is "give the API a callback and it will get called once in a while with samples" is terrible for multiple reasons, at least one of which is explained above. I talked to the SDL guys about this and to their credit they saw the problem immediately and SDL2 now has an application-pull way to get samples (I don't know how well it is supported on various platforms, or whether it is just a wrapper over the thread thing though, which would be Not Very Good.)

Re: Android’s 10 Millisecond Problem explained

#166
post #104

There is something I don't understand; maybe someone here can explain: Sound travels at about 340 m/s (in a typical room). That means it travels about 3.4 metres in 10 milliseconds. Therefore another way to get a 10 millisecond problem is to stand 3.4 metres from the orchestra. Most people sit farther than 3.4 metres from the orchestra, yet they don't complain about a lag between when the violin bow moves and the sou…

I'm a musician, 10ms latency would be fine, however as they note, most Android apps have 100ms latency, or 200ms round-trip latency. That is definitely not usable.

The funny part is that pipe organists would laugh at 100ms latency and say "cry me a river". With the pneumatics combined with the distance of the pipes from the performer, pipe organ latencies can be in the 200 to 500ms range. I asked my sister how she managed it and she said it was just a learned skill. She had to learn to completely tune out what she was hearing and play with the beat and music completely internalized.

The difference, of course, is that organists are usually not syncing up to other instruments. If there are other instruments involved, they tend to sync up with the organ.

Re: Android’s 10 Millisecond Problem explained

#167

Earlier quoted context omitted.

I am not sure why this is downvoted. Abstraction layers are often added to give flexibility and ease of use, sometimes at the cost of performance. For example ALSA has features like muxing together audio from several apps, while a lower level API might only allow one app to use audio.

iOS can mix audio while preserving low latency. It works much better because CoreAudio is basically a full audio routing and mixing engine and forces more realtime-ish requirements on implementations. The system level engine just takes audio from apps as if they were submixers. None of that stops you from having a high-level API on top, and in fact iOS has several at different levels of abstraction: AVAudioEngine giv…

Just came here to say that. The fact that core audio is a great performing API means even less performance critical stuff still gets the benefits of it even if the easy to use abstractions like AVAudioPlayer give up some flexibility and performance vs the native API. While abstraction can make things more flexible, I would argue that the main purpose is ease of use and hiding implementation details.

Re: Android’s 10 Millisecond Problem explained

#168

Earlier quoted context omitted.

Probably because they're not directly interacting with it, they're just watching it, which makes the delay a lot less noticeable. The 10ms delay is more of a problem for interactive apps where you eg. touch the screen and expect it to instantly make a sound. In this kind of feedback loop, even small delays are distracting and it becomes difficult to keep a beat because the lag is perceptible. If you tried to remotely…

Sort of like why audience clapping tends to get really messy until the performer does the "hands above the head" thing to get them back in sync.

Audiences have a natural tendency to slow down. They're mostly not musicians and I imagine their clapping as a response to the music (and the other clappers) rather than internalizing the music as a performer would.

Re: Android’s 10 Millisecond Problem explained

#169
post #165

Earlier quoted context omitted.

Please tell me what claim I made about CPU speeds is unsubstantiated nonsense?

I could type up a thorough explanation, but it would take about an hour, and I have a lot to do. It is actually not a bad idea to do such a write-up, but I don't think the appropriate venue for it is an ephemeral post on Hacker News ... I'd rather blog it somewhere that's more suitable for long-term reference. But I'll drop a few hints. First of all, nobody is talking about running interrupts at 48kHz. That is comple…

If you want a further analogy, it's like public transit. Which is a better commute: You take Bus A, which then drops you off at the stop for Bus B, at which you have to wait a varying and indeterminate amount of time, because the schedules for Bus A and Bus B are not synchronized; or just taking Bus C, that travels the same route without stopping?

Re: Android’s 10 Millisecond Problem explained

#170

Earlier quoted context omitted.

Yes, assembly is more flexible than my favorite higher level language. Because if you have access to assembly, you can implement the whatever higher level language or semantics you want. Whereas if you only have a higher level language, you have to work with/around the abstractions baked into it. For realtime applications it's better to at least have access to a low-level, less overhead API.

That is true. For example, in assembly we can create a higher level language with an absolutely air-tight, precisely tracing garbage collector that is free of issues like false retention. We cannot do that in C. That's because there are areas of the program state that are "off limits", and the compiler generates "GC ignorant" code.

In C you can create a higher level language with an absolutely air-tight, precisely tracing garbage collector that is free of issues like false retention.

You can do it portably if you just don't store objects on the C stack, and you can do it non-portably if you do so.

Post reply on HN