Audio-to-video synchronization

<h2 id="sources-of-error">Sources of error</h2>
AV-sync errors may accumulate across different stages, for a variety of reasons.
During creation, AV-sync errors may occur internally due to different <a href="/facts/Signal_processing/Npaja6zb">signal processing</a> delays between image and sound in <a href="/facts/Video_camera/pu84VbC7">video camera</a> and <a href="/facts/Microphone/7YWGlXCl">microphone</a>. The AV-sync delay is normally fixed. External AV-sync errors can occur if a microphone is placed far away from the sound source, the audio will be out of sync because the <a href="/facts/Speed_of_sound/dYtbt1nf">speed of sound</a> is much lower than the <a href="/facts/Speed_of_light/sKSQwdNx">speed of light</a>. If the sound source is 340 meters from the microphone, then the sound arrives approximately 1 second later than the light. The AV-sync delay increases with distance. During mixing of video clips normally either the audio or video needs to be delayed so they are synchronized. The AV-sync delay is static but can vary with the individual clip. <a href="/facts/Video_editing/VoWFUpIE">Video editing</a> effects can delay video causing it to lag the audio.
Transmission (<a href="/facts/Broadcasting/AV1f3dGy">broadcasting</a>), reception, and playback may also introduce AV-sync errors. A video camera with built-in microphones or line-in may not delay sound and video paths by the same amount. Solid-state video cameras (e.g. <a href="/facts/Charge-coupled_device/l6xffIcz">charge-coupled device</a> (CCD) and <a href="/facts/CMOS_image_sensor/6Trt4SWj">CMOS image sensors</a>) can delay the video signal by one or more frames. Audio and video signal processing circuitry exists with significant (and potentially non-constant) delays in television systems. Frame synchronizers, digital video effects processors, video noise reduction, format converters, and <a href="/facts/Video_compression/gMFuY4FT">compression systems</a> are examples of widely used signal-processing elements that may contribute significant video delay.
Processing circuits <a href="/facts/Format_conversion/ovcQxA3K">format conversion</a> and deinterlace processing in video monitors can add one or more frames of video delay. A video monitor with built-in speakers or line-out may not delay sound and video paths equally. Some video monitors contain internal user-adjustable audio delays to aid in correction of errors. 
Some transmission protocols like <a href="/facts/Real-time_Transport_Protocol/YKQK1RK2">RTP</a> require an out-of-band method for synchronizing media streams. In some RTP systems, each media stream has its own timestamp using an independent clock rate and per-stream randomized starting value. A <a href="/facts/RTCP/grCfaxxe">RTCP</a> <a href="/facts/RTP_Control_Protocol/grCfaxxe">Sender Report (SR)</a> may be needed for each stream in order to synchronize streams.<a class="footnote-ref" id="fnref:3" href="#fn:3">3</a>

<h2 id="effect-of-no-explicit-av-sync-timing">Effect of no explicit AV-sync timing</h2>
When a digital or analog AV system stream does not have a synchronization method or mechanism, the stream may become out of sync. In film movies these timing errors are most commonly caused by worn films skipping over the <a href="/facts/Movie_projector/SXzyFeDQ">movie projector</a> sprockets because the film has torn sprocket holes. Errors can also be caused by the <a href="/facts/Projectionist/hlXv9se1">projectionist</a> misthreading the film in the projector. 
Synchronization errors have become a significant problem in the <a href="/facts/Digital_television/KQCyILQa">digital television</a> industry because of the use of large amounts of video signal processing in television production, television broadcasting and <a href="/facts/Pixelated/2dcH7BWU">pixelated</a> television displays such as <a href="/facts/LCD/8B6xGBEz">LCD</a>, <a href="/facts/Digital_Light_Processing/jxMs8emm">DLP</a> and <a href="/facts/Plasma_displays/1RqxrYBo">plasma displays</a>. Pixelated displays utilize complex video signal processing to convert the resolution of the incoming video signal to the native resolution of the pixelated display, for example converting standard definition video to be displayed on a high definition display. Synchronization problems are commonly caused when significant amounts of <a href="/facts/Video_processing/n08sasRV">video processing</a> is performed on the video part of the television program. Typical sources of significant video delays in the television field include video synchronizers and video compression encoders and decoders. Particularly troublesome encoders and decoders are used in <a href="/facts/MPEG/ygmT7JBB">MPEG</a> compression systems utilized for broadcasting <a href="/facts/Digital_television/KQCyILQa">digital television</a> and storing television programs on consumer and professional recording and playback devices.
In broadcast television, it is not unusual for lip-sync error to vary by over 100 ms (several video frames) from time to time. AV-sync is commonly corrected and maintained with an <a href="/facts/Audio_synchronizer/n9L4DcK1">audio synchronizer</a>. Television industry standards organizations have established acceptable amounts of audio and video timing error and suggested practices related to maintaining acceptable timing.<a class="footnote-ref" id="fnref:4" href="#fn:4">4</a><a class="footnote-ref" id="fnref:5" href="#fn:5">5</a> The EBU Recommendation R37 "The relative timing of the sound and vision components of a television signal" states that end-to-end audio/video sync should be within +40 ms and -60 ms (audio before/after video, respectively) and that each stage should be within +5 ms and -15 ms.<a class="footnote-ref" id="fnref:6" href="#fn:6">6</a>

<h2 id="viewer-experience-of-incorrectly-synchronized-av-sync">Viewer experience of incorrectly synchronized AV-sync</h2>
The result typically leaves a filmed or televised character's mouth movements mismatching spoken dialog, hence the term lip flap or lip-sync error. The resulting audio-video sync error can be annoying to the viewer and may even cause the viewer to not enjoy the program, decrease the effectiveness of the program or lead to a negative perception of the speaker on the part of the viewer.<a class="footnote-ref" id="fnref:7" href="#fn:7">7</a> The potential loss of effectiveness is of particular concern for product commercials and political candidates. Television industry standards organizations, such as the <a href="/facts/Advanced_Television_Systems_Committee/JhzeErLa">Advanced Television Systems Committee</a>, have become involved in setting standards for audio-video sync errors.<a class="footnote-ref" id="fnref:8" href="#fn:8">8</a>
Because of these annoyances, AV-sync error is a concern to the television programming industry, including television stations, networks, advertisers and program production companies. Unfortunately, the advent of high-definition flat-panel display technologies (LCD, DLP and plasma), which can delay video more than audio, has moved the problem into the viewer's home and beyond the control of the television programming industry alone. Consumer product companies now offer audio-delay adjustments to compensate for video-delay changes in TVs, <a href="/facts/Soundbar/7g4ulSxQ">soundbars</a> and A/V receivers,<a class="footnote-ref" id="fnref:9" href="#fn:9">9</a> and several companies manufacture dedicated digital audio delays made exclusively for lip-sync error correction.

<h2 id="recommendations">Recommendations</h2>
For television applications, the <a href="/facts/Advanced_Television_Systems_Committee/JhzeErLa">Advanced Television Systems Committee</a> recommends that audio should lead video by no more than 15 <a href="/facts/Millisecond/xHCyv235">ms</a> and audio should lag video by no more than 45 ms.<a class="footnote-ref" id="fnref:10" href="#fn:10">10</a> However, the <a href="/facts/ITU/Yn8UKKkw">ITU</a> performed strictly <a href="/facts/Controlled_test/6ALSxX8M">controlled tests</a> with expert viewers and found that the threshold for detectability is 45 ms lead to 125 ms lag.<a class="footnote-ref" id="fnref:11" href="#fn:11">11</a> For film, acceptable lip sync is considered to be no more than 22 milliseconds in either direction.<a class="footnote-ref" id="fnref:12" href="#fn:12">12</a><a class="footnote-ref" id="fnref:13" href="#fn:13">13</a>
The <a href="/facts/Consumer_Electronics_Association/4mwfDzM8">Consumer Electronics Association</a> has published a set of recommendations for how digital television receivers should implement A/V sync.<a class="footnote-ref" id="fnref:14" href="#fn:14">14</a>

<h2 id="smpte-st2064">SMPTE ST2064</h2>
<a href="/facts/SMPTE/84NgELfP">SMPTE</a> standard ST2064, published in 2015,<a class="footnote-ref" id="fnref:15" href="#fn:15">15</a> provides technology to reduce or eliminate lip-sync errors in digital television. The standard utilizes audio and video fingerprints taken from a television program. The fingerprints can be recovered and used to correct the accumulated lip-sync error. When fingerprints have been generated for a TV program, and the required technology is incorporated, the viewer's <a href="/facts/Television_set/8moTsqhB">television set</a> has the ability to continuously measure and correct lip-sync errors.<a class="footnote-ref" id="fnref:16" href="#fn:16">16</a><a class="footnote-ref" id="fnref:17" href="#fn:17">17</a>

<h2 id="timestamps">Timestamps</h2>
<a href="/facts/Presentation_time_stamp/VJgrHw5K">Presentation time stamps</a> (PTS) are embedded in <a href="/facts/MPEG_transport_stream/2wBLnyeD">MPEG transport streams</a> to precisely signal when each audio and video segment is to be presented and avoid AV-sync errors. However, these timestamps are often added after the video undergoes frame synchronization, format conversion and preprocessing, and thus the lip sync errors created by these operations will not be corrected by the addition and use of timestamps.<a class="footnote-ref" id="fnref:18" href="#fn:18">18</a><a class="footnote-ref" id="fnref:19" href="#fn:19">19</a><a class="footnote-ref" id="fnref:20" href="#fn:20">20</a><a class="footnote-ref" id="fnref:21" href="#fn:21">21</a>
The <a href="/facts/Real-time_Transport_Protocol/YKQK1RK2">Real-time Transport Protocol</a> clocks media using origination <a href="/facts/Timestamps/6lApANE6">timestamps</a> on an arbitrary timeline. A real-time clock such as one delivered by the <a href="/facts/Network_Time_Protocol/vS21KqT5">Network Time Protocol</a> or <a href="/facts/Precision_Time_Protocol/N5u5HVqx">Precision Time Protocol</a> and described in the <a href="/facts/Session_Description_Protocol/WMnnA5jb">Session Description Protocol</a><a class="footnote-ref" id="fnref:22" href="#fn:22">22</a> associated with the media may be used to synchronize media. A server may then be used for synchronization between multiple receivers.<a class="footnote-ref" id="fnref:23" href="#fn:23">23</a>

<h2 id="see-also">See also</h2>
<ul><li><a href="/facts/Clapperboard/dQKfNlHx">Clapperboard</a></li>
<li><a href="/facts/Dubbing/wXdfmxl4">Dubbing</a></li>
<li><a href="/facts/Lag_(video_games)/kb0QfGzX">Lag (video games)</a></li></ul>

<h2 id="further-reading">Further reading</h2>
<ul><li>Cugnini, Aldo (Sep 1, 2007). <a href="https://web.archive.org/web/20151008114533/http://www.tvtechnology.com/media-systems/0191/managing-lip-sync/255548">"Managing lip sync"</a>. TV Technology, originally from Broadcast Engineering. Archived from <a href="http://www.tvtechnology.com/media-systems/0191/managing-lip-sync/255548">the original</a> on October 8, 2015. Retrieved 2008-10-19.</li>
<li>R.A. Salmon; Andrew Mason (January 2009). <a href="https://web.archive.org/web/20150913111322/http://downloads.bbc.co.uk/rd/pubs/whp/whp-pdf-files/WHP176.pdf">"Factors affecting perception of audio-video synchronisation in television"</a> (PDF). <a href="/facts/BBC_Research_%2526_Development/quEKy1jS">BBC Research & Development</a>. Archived from <a href="https://www.bbc.co.uk/rd/publications/whitepaper176">the original</a> on 2015-09-13.</li>
<li>Sieranoja, S.; Sahidullah, Md; Kinnunen, T.; Komulainen, J.; Hadid, A. (July 2018). <a href="http://cs.joensuu.fi/pages/tkinnu/webpage/pdf/audiovisual_synchrony_2018.pdf">"Audiovisual Synchrony Detection with Optimized Audio Features"</a> (PDF). <a href="http://urn.fi/urn:nbn:fi-fe2020041415345">2018 IEEE 3rd International Conference on Signal and Image Processing (ICSIP)</a>. pp. 377–381. <a href="/facts/Doi_(identifier)/muM9Etpq">doi</a>:<a href="https://doi.org/10.1109%2FSIPROCESS.2018.8600424">10.1109/SIPROCESS.2018.8600424</a>. <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-1-5386-6396-7. <a href="/facts/S2CID_(identifier)/ldJsHa2Y">S2CID</a> <a href="https://api.semanticscholar.org/CorpusID:51682024">51682024</a>.</li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1">"ITU-R BT.1359-1, Relative Timing of Sound and Vision for Broadcasting" (PDF). ITU. 1998. Retrieved 30 May 2015. <a href="https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.1359-1-199811-I!!PDF-E.pdf" target="_blank">https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.1359-1-199811-I!!PDF-E.pdf</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></li>
<li id="fn:2">Patrick Waddell; Graham Jones; Adam Goldberg. "Audio/Video Standards and Solutions A Status Report" (PDF). ATSC. Archived from the original (PDF) on 17 February 2016. Retrieved 4 April 2012. <a href="https://web.archive.org/web/20160217123737/http://atsc.org/wp-content/uploads/pdf/audio_seminar/12%20-%20JONES%20-%20Audio%20and%20Video%20synchronization-Status.pdf" target="_blank">https://web.archive.org/web/20160217123737/http://atsc.org/wp-content/uploads/pdf/audio_seminar/12%20-%20JONES%20-%20Audio%20and%20Video%20synchronization-Status.pdf</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></li>
<li id="fn:3">RFC 3550 <a href="/wiki/RFC_(identifier)" target="_blank">/wiki/RFC_(identifier)</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></li>
<li id="fn:4">IS-191: Relative Timing of Sound and Vision for Broadcast Operations, ATSC, 2003-06-26, archived from the original on 2012-03-21 <a href="https://web.archive.org/web/20120321084955/http://www.atsc.org/cms/index.php/standards/document-download/doc_download/47-is-191-relative-timing-of-sound-and-vision-for-broadcast-operations" target="_blank">https://web.archive.org/web/20120321084955/http://www.atsc.org/cms/index.php/standards/document-download/doc_download/47-is-191-relative-timing-of-sound-and-vision-for-broadcast-operations</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></li>
<li id="fn:5">"ITU-R BT.1359-1, Relative Timing of Sound and Vision for Broadcasting" (PDF). ITU. 1998. Retrieved 30 May 2015. <a href="https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.1359-1-199811-I!!PDF-E.pdf" target="_blank">https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.1359-1-199811-I!!PDF-E.pdf</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></li>
<li id="fn:6">"The relative timing of the sound and vision components of a television signal" (PDF). <a href="http://tech.ebu.ch/docs/r/r037.pdf" target="_blank">http://tech.ebu.ch/docs/r/r037.pdf</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></li>
<li id="fn:7">Byron Reeves; David Voelker (October 1993). "Effects of Audio-Video Asynchrony on Viewer's Memory, Evaluation of Content and Detection Ability" (PDF). Archived from the original (PDF) on 2 October 2008. Retrieved 2008-10-19. <a href="https://web.archive.org/web/20081002172843/http://www.pixelinstruments.tv/pdf/Articles/Effects%20of%20Audio-Video%20Asynchrony.PDF" target="_blank">https://web.archive.org/web/20081002172843/http://www.pixelinstruments.tv/pdf/Articles/Effects%20of%20Audio-Video%20Asynchrony.PDF</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></li>
<li id="fn:8">IS-191: Relative Timing of Sound and Vision for Broadcast Operations, ATSC, 2003-06-26, archived from the original on 2012-03-21 <a href="https://web.archive.org/web/20120321084955/http://www.atsc.org/cms/index.php/standards/document-download/doc_download/47-is-191-relative-timing-of-sound-and-vision-for-broadcast-operations" target="_blank">https://web.archive.org/web/20120321084955/http://www.atsc.org/cms/index.php/standards/document-download/doc_download/47-is-191-relative-timing-of-sound-and-vision-for-broadcast-operations</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></li>
<li id="fn:9">"Lip-sync error: Causes, solutions". Retrieved 2024-06-13. <a href="https://www.cnet.com/tech/home-entertainment/lip-sync-error-causes-solutions/" target="_blank">https://www.cnet.com/tech/home-entertainment/lip-sync-error-causes-solutions/</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></li>
<li id="fn:10">IS-191: Relative Timing of Sound and Vision for Broadcast Operations, ATSC, 2003-06-26, archived from the original on 2012-03-21 <a href="https://web.archive.org/web/20120321084955/http://www.atsc.org/cms/index.php/standards/document-download/doc_download/47-is-191-relative-timing-of-sound-and-vision-for-broadcast-operations" target="_blank">https://web.archive.org/web/20120321084955/http://www.atsc.org/cms/index.php/standards/document-download/doc_download/47-is-191-relative-timing-of-sound-and-vision-for-broadcast-operations</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></li>
<li id="fn:11">"ITU-R BT.1359-1, Relative Timing of Sound and Vision for Broadcasting" (PDF). ITU. 1998. Retrieved 30 May 2015. <a href="https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.1359-1-199811-I!!PDF-E.pdf" target="_blank">https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.1359-1-199811-I!!PDF-E.pdf</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></li>
<li id="fn:12">"The relative timing of the sound and vision components of a television signal" (PDF). <a href="http://tech.ebu.ch/docs/r/r037.pdf" target="_blank">http://tech.ebu.ch/docs/r/r037.pdf</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></li>
<li id="fn:13">Sara Kudrle; et al. (July 2011). "Fingerprinting for Solving A/V Synchronization Issues within Broadcast Environments". Motion Imaging Journal. SMPTE. Appropriate A/V sync limits have been established and the range that is considered acceptable for film is +/- 22 ms. The range for video, according to the ATSC, is up to 15 ms lead time and about 45 ms lag time <a href="/wiki/SMPTE" target="_blank">/wiki/SMPTE</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></li>
<li id="fn:14">Consumer Electronics Association. "CEA-CEB20 R-2013: A/V Synchronization Processing Recommended Practice". Archived from the original on 2015-05-30. <a href="https://web.archive.org/web/20150530162633/http://www.ce.org/Standards/Standard-Listings/R4-Video-Systems-Committee/CEA-CEB20.aspx" target="_blank">https://web.archive.org/web/20150530162633/http://www.ce.org/Standards/Standard-Listings/R4-Video-Systems-Committee/CEA-CEB20.aspx</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></li>
<li id="fn:15">ST 2064:2015 - SMPTE Standard - Audio to Video Synchronization Measurement, SMPTE, 2015 <a href="https://ieeexplore.ieee.org/search/searchresult.jsp?newsearch=true&queryText=SMPTE%20ST2064" target="_blank">https://ieeexplore.ieee.org/search/searchresult.jsp?newsearch=true&queryText=SMPTE%20ST2064</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></li>
<li id="fn:16">SMPTE Standards Update: The Lip-Sync Challenge, SMPTE, 10 December 2013, archived from the original on 2021-12-15 <a href="https://www.youtube.com/watch?v=4C21mUQ0UB4" target="_blank">https://www.youtube.com/watch?v=4C21mUQ0UB4</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></li>
<li id="fn:17">SMPTE Standards Update: The Lip-Sync Challenge (PDF), SMPTE, 10 December 2013, archived from the original (PDF) on 2016-08-26, retrieved 2016-06-09 <a href="https://web.archive.org/web/20160826112534/https://www.smpte.org/sites/default/files/2013-12-10-Lip-Sync-Briscoe-V2-Handout.pdf" target="_blank">https://web.archive.org/web/20160826112534/https://www.smpte.org/sites/default/files/2013-12-10-Lip-Sync-Briscoe-V2-Handout.pdf</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></li>
<li id="fn:18">"MPEG-2 Systems FAQ: 19. Where are the PTSs and DTSs inserted?". Archived from the original on 2008-07-26. Retrieved 2007-12-27. <a href="https://web.archive.org/web/20080726181029/http://www.chiariglione.org/mpeg/faq/mp2-sys/mp2-sys.htm#mp2-19" target="_blank">https://web.archive.org/web/20080726181029/http://www.chiariglione.org/mpeg/faq/mp2-sys/mp2-sys.htm#mp2-19</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></li>
<li id="fn:19">Arpi (7 May 2003). "MPlayer-G2-dev: mpeg container's timing (PTS values)". <a href="http://lists.mplayerhq.hu/pipermail/mplayer-g2-dev/2003-May/000004.html" target="_blank">http://lists.mplayerhq.hu/pipermail/mplayer-g2-dev/2003-May/000004.html</a> <a href="#fnref:19" class="footnote-back-ref">↩</a></li>
<li id="fn:20">"birds-eye.net: DTS - Decode Time Stamp". <a href="http://www.birds-eye.net/definition/d/dts-decode_time_stamp.shtml" target="_blank">http://www.birds-eye.net/definition/d/dts-decode_time_stamp.shtml</a> <a href="#fnref:20" class="footnote-back-ref">↩</a></li>
<li id="fn:21">"SVCD2DVD: Author and burn DVDs: AVI to DVD, DivX to DVD, Xvid to DVD, MPEG to DVD, SVCD to DVD, VCD to DVD, PAL to NTSC conversion, HDTV2DVD, HDTV to DVD, BLURAY". www.svcd2dvd.com. Archived from the original on March 14, 2004. <a href="https://web.archive.org/web/20040314053333/http://www.svcd2dvd.com/Guides/AVSync/default.aspx" target="_blank">https://web.archive.org/web/20040314053333/http://www.svcd2dvd.com/Guides/AVSync/default.aspx</a> <a href="#fnref:21" class="footnote-back-ref">↩</a></li>
<li id="fn:22">A. Williams; K. Gross; et al. (June 2014). RTP Clock Source Signalling. Internet Engineering Task Force. doi:10.17487/RFC7273. RFC 7273. Proposed Standard.
 <a href="https://datatracker.ietf.org/doc/html/rfc7273" target="_blank">https://datatracker.ietf.org/doc/html/rfc7273</a> <a href="#fnref:22" class="footnote-back-ref">↩</a></li>
<li id="fn:23">R. van Brandenburg; et al. (June 2014). Inter-Destination Media Synchronization (IDMS) Using the RTP Control Protocol (RTCP). Internet Engineering Task Force. doi:10.17487/RFC7272. RFC 7272. Proposed Standard.
 <a href="https://datatracker.ietf.org/doc/html/rfc7272" target="_blank">https://datatracker.ietf.org/doc/html/rfc7272</a> <a href="#fnref:23" class="footnote-back-ref">↩</a></li>
</ol>

Audio-to-video synchronization open-in-new

Audio-to-video synchronization