Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Android’s 10 Millisecond Problem explained

superpowered.com

Post reply on HN
Print view View original

181–190 of 311 posts

Re: Android’s 10 Millisecond Problem explained

#181

Post by wtallis »

daddykotex wrote: post #137
S_A_P wrote: post #43

Reading about this takes me back to the late 90s. I had just gotten serious about recording music on my computer and purchased an 866mhz dell pc. I was a using windows 98 and ASIO was still mostly a Steinberg only development and I couldn't afford cubase. I got a hand me down version of cakewalk pro audio 8. I remember the latency of that set up was about 35ms and thinking this is pretty good. There was some degree o…

Is it possible that their Audio API is much simpler due to the fact that they control all of the implementations (Hardware interface) of iOS?

Not really. It's the same API they've been using on Macs since the PowerPC days, so they've implemented it for a bunch of different audio chips over the years and that includes a lot of common off-the-shelf solutions. The Hackintosh scene provides fully-capable drivers for all the audio chips you're likely to find in a PC today. Hardware selection or effort put into each driver are almost certainly not factors.

Re: Android’s 10 Millisecond Problem explained

#183

Post by ianlevesque »

daddykotex wrote: post #137
S_A_P wrote: post #43

Reading about this takes me back to the late 90s. I had just gotten serious about recording music on my computer and purchased an 866mhz dell pc. I was a using windows 98 and ASIO was still mostly a Steinberg only development and I couldn't afford cubase. I got a hand me down version of cakewalk pro audio 8. I remember the latency of that set up was about 35ms and thinking this is pretty good. There was some degree o…

Is it possible that their Audio API is much simpler due to the fact that they control all of the implementations (Hardware interface) of iOS?

With Android the variety of customized android builds on various manufacturers does make this more challenging than it needs to be. As the article points out, many vendors simply break realtime audio by using slow drivers and slow paths.

However, as someone who has dealt with Android, iOS, Windows Phone, and BB audio APIs, Android's audio API really is just amateur hour compared to Apple's, even to the present day. In older versions it was simply unusable for anything more complicated than playing a sound effect or fixed-length mp3 file. Today it functions for more use cases but still abstracts concepts (like codecs and container parsing) in flawed and inconvenient places.

Windows Phone and BB are still even worse though.

Re: Android’s 10 Millisecond Problem explained

#184

Post by cnvogel »

jblow wrote: post #179
nitrogen wrote: post #174

Earlier quoted context omitted.

The other commenter was talking about audio software that both consumes and produces samples at a fixed rate. Clearly, if the audio software is late grabbing 64 samples from the input device, it's also late delivering the next 64 to the output device, and there will be a dropout. The output sample clock has to be the timing master, and the software can never be late, and since it's also waiting for the input audio, i…

I am not sure we can make the assumption that the input and output devices are on the same clocks or run at the same rates. Maybe they are (in a good system you'd hope they would be), but I can think of a lot of cases where that wouldn't be true. However, even when they are synced, you can still easily see the problem. The software is never going to be able to do its job in zero time, so we always take a delay of at…

Music or video production studios typically have a central clock, so for this use-case the sample rates should be perfect. But even if the input and output devices are on perfect clocks, with NTSC (59.94 Hz), you'd need a very odd number of samples per video frame in your software, if your processing would happen at a integer fraction of the video frame rate.

Re: Android’s 10 Millisecond Problem explained

#185

Post by coreyoconnor »

afandian wrote: post #2

Anyone with any insight about how Apple does this?

One aspect of CoreAudio that, IMO, contributes to low latency: CoreAudio provides the buffers to process to the AudioUnit. Not the other way around. This permits CoreAudio to chain AudioUnits while minimizing buffer use. In the ideal case where all AudioUnits support the same audio stream format they are all provided the same buffers to process. Which can get close to zero-copy, zero-allocation audio rendering.

I don't know enough about android to say that this is the key aspect tho. Perhaps the low level API of android is the same? I didn't find an easy reference in a quick search.

Re: Android’s 10 Millisecond Problem explained

#186

Post by imperialdrive »

IvyMike wrote: post #156
squeaky-clean wrote: post #143

Earlier quoted context omitted.

The audience doesn't need instant feedback and low latency, the performers do. If you listen to a classic rock album, you're hearing the sounds with a "latency" decades after they've been played. But the playing is cohesive and tight. If one of the guitarists was consistently 50ms off, you would notice it. > Most Android apps have more than 100 ms of audio output latency, and more than 200 ms of round-trip (audio inp…

> If one of the guitarists was consistently 50ms off, you would notice it. The best movie about this, ever: https://youtu.be/VnuImW1dWAk

that really is one of the top movies in recent history - like a shawshank for musicians...

Re: Android’s 10 Millisecond Problem explained

#187

Post by kabdib »

weaksauce wrote: post #178
kabdib wrote: post #120

I'd heard over the years that working with isochronous systems was difficult. I'd done a number of real-time systems before, and written OS schedulers and NTP-like systems and so forth. A little audio work should be a walk in the park, right? A little manly-man programming from the wrist and we move on to real problems. So I walked into an audio project thinking that "Oh, this latency and synchronization stuff, how b…

A real postmortem of this would be a fascinating read.

Well, if something bad happens I might have to work there again, so . . . not for a few years, probably :-)

Re: Android’s 10 Millisecond Problem explained

#188

Post by kazinator »

aidenn0 wrote: post #170
kazinator wrote: post #97

Earlier quoted context omitted.

That is true. For example, in assembly we can create a higher level language with an absolutely air-tight, precisely tracing garbage collector that is free of issues like false retention. We cannot do that in C. That's because there are areas of the program state that are "off limits", and the compiler generates "GC ignorant" code.

In C you can create a higher level language with an absolutely air-tight, precisely tracing garbage collector that is free of issues like false retention. You can do it portably if you just don't store objects on the C stack, and you can do it non-portably if you do so.

> In C you can create a higher level language with an absolutely air-tight, precisely tracing garbage collector that is free of issues like false retention.

If so, you will be the first; I look forward to the "Show HN:" when it's done.

> You can do it portably if you just don't store objects on the C stack,

Sure, for example, you can do it portably in C if you write a complete emulator for an 80386, and then use that to run an assembly language program. That program isn't a utility for your C code; it's not extending the host C with a garbage collector or whatever else.

If you do not store object references on the stack, your use of C is severely crippled to the point that it's not really C any more. For one thing, C function arguments are on the stack (or, more abstractly, "automatic storage"), so say goodbye to conventional use of C argument passing: the backbone of most normal C programming.

That garbage collector isn't for C. C has automatic storage, and a proper garbage collector has to traverse it.

> and you can do it non-portably if you do so.

Sure, if non-portably means going as far as forking a specific C compiler with your custom hacks, and requiring that C compiler, or else living with the imprecision and taking a whack-a-mole approach to plugging the issues as they arise.

Re: Android’s 10 Millisecond Problem explained

#189

Post by aidenn0 »

kazinator wrote: post #188
aidenn0 wrote: post #170

Earlier quoted context omitted.

In C you can create a higher level language with an absolutely air-tight, precisely tracing garbage collector that is free of issues like false retention. You can do it portably if you just don't store objects on the C stack, and you can do it non-portably if you do so.

> In C you can create a higher level language with an absolutely air-tight, precisely tracing garbage collector that is free of issues like false retention. If so, you will be the first; I look forward to the "Show HN:" when it's done. > You can do it portably if you just don't store objects on the C stack, Sure, for example, you can do it portably in C if you write a complete emulator for an 80386, and then use that…

Chicken scheme uses C function calls, and merely requires a contiguous stack for automatic storage (which is technically non-portable, but doesn't require forking a compiler):

http://en.wikipedia.org/wiki/CHICKEN_%28Scheme_implementatio...

[edit] And here is a toy scheme interpreter that uses precise garbage collection to show off the Ravenbrook MPS

http://www.ravenbrook.com/project/mps/master/manual/html/gui...

Re: Android’s 10 Millisecond Problem explained

#190

Post by fnordfnordfnord »

__david__ wrote: post #166
Synaesthesia wrote: post #113

Earlier quoted context omitted.

I'm a musician, 10ms latency would be fine, however as they note, most Android apps have 100ms latency, or 200ms round-trip latency. That is definitely not usable.

The funny part is that pipe organists would laugh at 100ms latency and say "cry me a river". With the pneumatics combined with the distance of the pipes from the performer, pipe organ latencies can be in the 200 to 500ms range. I asked my sister how she managed it and she said it was just a learned skill. She had to learn to completely tune out what she was hearing and play with the beat and music completely internal…

The latency has to be predictable and for most people probably also consistent.
Post reply on HN