135f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Lypage.title=Audio Latency 235f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly@jd:body 335f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<!-- 5768b82a9dfbdd8504eae2736d283a60a37c7a547Clay Murphy Copyright 2013 The Android Open Source Project 635f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 735f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly Licensed under the Apache License, Version 2.0 (the "License"); 835f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly you may not use this file except in compliance with the License. 935f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly You may obtain a copy of the License at 1035f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 1135f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly http://www.apache.org/licenses/LICENSE-2.0 1235f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 1335f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly Unless required by applicable law or agreed to in writing, software 1435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly distributed under the License is distributed on an "AS IS" BASIS, 1535f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 1635f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly See the License for the specific language governing permissions and 1735f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly limitations under the License. 1835f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly--> 1935f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<div id="qv-wrapper"> 2035f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <div id="qv"> 2135f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <h2>In this document</h2> 2235f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <ol id="auto-toc"> 2335f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly </ol> 2435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly </div> 2535f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</div> 2635f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 2735f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<p>Audio latency is the time delay as an audio signal passes through a system. 2835f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly For a complete description of audio latency for the purposes of Android 29c28f237dd66f3826b36b51dbdea57411b9585869Clay Murphy compatibility, see <em>Section 5.5 Audio Latency</em> 3035f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly in the <a href="http://source.android.com/compatibility/index.html">Android CDD</a>. 31c28f237dd66f3826b36b51dbdea57411b9585869Clay Murphy See <a href="latency_design.html">Design For Reduced Latency</a> for an 32c28f237dd66f3826b36b51dbdea57411b9585869Clay Murphy understanding of Android's audio latency-reduction efforts. 3335f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</p> 3435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 3535f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<p> 3647b1d3f8ffd20344e0d17eac97a9ceee2108b3a5Clay Murphy This page focuses on the contributors to output latency, 3735f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly but a similar discussion applies to input latency. 3835f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</p> 3935f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<p> 40c28f237dd66f3826b36b51dbdea57411b9585869Clay Murphy Assuming the analog circuitry does not contribute significantly, then the major 41c28f237dd66f3826b36b51dbdea57411b9585869Clay Murphysurface-level contributors to audio latency are the following: 4235f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</p> 4335f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 4435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<ul> 4535f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <li>Application</li> 4635f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <li>Total number of buffers in pipeline</li> 4735f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <li>Size of each buffer, in frames</li> 4835f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <li>Additional latency after the app processor, such as from a DSP</li> 4935f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</ul> 5035f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 5135f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<p> 5235f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly As accurate as the above list of contributors may be, it is also misleading. 5335f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly The reason is that buffer count and buffer size are more of an 5435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <em>effect</em> than a <em>cause</em>. What usually happens is that 5535f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly a given buffer scheme is implemented and tested, but during testing, an audio 56c28f237dd66f3826b36b51dbdea57411b9585869Clay Murphy underrun is heard as a "click" or "pop." To compensate, the 5735f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly system designer then increases buffer sizes or buffer counts. 5835f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly This has the desired result of eliminating the underruns, but it also 5935f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly has the undesired side effect of increasing latency. 6035f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</p> 6135f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 6235f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<p> 63c28f237dd66f3826b36b51dbdea57411b9585869Clay Murphy A better approach is to understand the causes of the 6435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly underruns and then correct those. This eliminates the 6535f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly audible artifacts and may even permit even smaller or fewer buffers 6635f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly and thus reduce latency. 6735f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</p> 6835f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 6935f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<p> 7035f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly In our experience, the most common causes of underruns include: 7135f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</p> 7235f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<ul> 7335f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <li>Linux CFS (Completely Fair Scheduler)</li> 7435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <li>high-priority threads with SCHED_FIFO scheduling</li> 7535f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <li>long scheduling latency</li> 7635f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <li>long-running interrupt handlers</li> 7735f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <li>long interrupt disable time</li> 7835f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</ul> 7935f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 8035f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<h3>Linux CFS and SCHED_FIFO scheduling</h3> 8135f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<p> 8235f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly The Linux CFS is designed to be fair to competing workloads sharing a common CPU 8335f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly resource. This fairness is represented by a per-thread <em>nice</em> parameter. 8435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly The nice value ranges from -19 (least nice, or most CPU time allocated) 8535f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly to 20 (nicest, or least CPU time allocated). In general, all threads with a given 8635f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly nice value receive approximately equal CPU time and threads with a 8735f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly numerically lower nice value should expect to 8835f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly receive more CPU time. However, CFS is "fair" only over relatively long 8935f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly periods of observation. Over short-term observation windows, 9035f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly CFS may allocate the CPU resource in unexpected ways. For example, it 9135f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly may take the CPU away from a thread with numerically low niceness 9235f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly onto a thread with a numerically high niceness. In the case of audio, 9335f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly this can result in an underrun. 9435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</p> 9535f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 9635f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<p> 9735f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly The obvious solution is to avoid CFS for high-performance audio 98c28f237dd66f3826b36b51dbdea57411b9585869Clay Murphy threads. Beginning with Android 4.1, such threads now use the 9935f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <code>SCHED_FIFO</code> scheduling policy rather than the <code>SCHED_NORMAL</code> (also called 10035f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <code>SCHED_OTHER</code>) scheduling policy implemented by CFS. 10135f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</p> 10235f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 10335f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<p> 10435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly Though the high-performance audio threads now use <code>SCHED_FIFO</code>, they 10535f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly are still susceptible to other higher priority <code>SCHED_FIFO</code> threads. 10635f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly These are typically kernel worker threads, but there may also be a few 10735f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly non-audio user threads with policy <code>SCHED_FIFO</code>. The available <code>SCHED_FIFO</code> 10835f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly priorities range from 1 to 99. The audio threads run at priority 10935f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 2 or 3. This leaves priority 1 available for lower priority threads, 110c28f237dd66f3826b36b51dbdea57411b9585869Clay Murphy and priorities 4 to 99 for higher priority threads. We recommend 11135f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly you use priority 1 whenever possible, and reserve priorities 4 to 99 for 11235f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly those threads that are guaranteed to complete within a bounded amount 113c28f237dd66f3826b36b51dbdea57411b9585869Clay Murphy of time and are known to not interfere with scheduling of audio threads. 11435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</p> 11535f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 11635f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<h3>Scheduling latency</h3> 11735f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<p> 11835f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly Scheduling latency is the time between when a thread becomes 11935f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly ready to run, and when the resulting context switch completes so that the 120c28f237dd66f3826b36b51dbdea57411b9585869Clay Murphy thread actually runs on a CPU. The shorter the latency the better, and 12135f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly anything over two milliseconds causes problems for audio. Long scheduling 12235f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly latency is most likely to occur during mode transitions, such as 12335f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly bringing up or shutting down a CPU, switching between a security kernel 12435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly and the normal kernel, switching from full power to low-power mode, 12535f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly or adjusting the CPU clock frequency and voltage. 12635f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</p> 12735f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 12835f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<h3>Interrupts</h3> 12935f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<p> 13035f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly In many designs, CPU 0 services all external interrupts. So a 13135f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly long-running interrupt handler may delay other interrupts, in particular 132c28f237dd66f3826b36b51dbdea57411b9585869Clay Murphy audio direct memory access (DMA) completion interrupts. Design interrupt handlers 13335f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly to finish quickly and defer any lengthy work to a thread (preferably 13435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly a CFS thread or <code>SCHED_FIFO</code> thread of priority 1). 13535f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</p> 13635f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 13735f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly<p> 13835f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly Equivalently, disabling interrupts on CPU 0 for a long period 13935f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly has the same result of delaying the servicing of audio interrupts. 14035f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly Long interrupt disable times typically happen while waiting for a kernel 14135f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly <i>spin lock</i>. Review these spin locks to ensure that 14235f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly they are bounded. 14335f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly</p> 14435f2fda6aaeaf733ab68a3b7f7ccc67f009c09a9Robert Ly 145