Example tutorial: The lip-sync problem in digital television: a demonstration and solution

Do you find on your TV that people’s lips move after you have heard the words spoken? This is the lip-sync problem. There are several causes, and several solutions.

The video demonstrates the lip sync problem and presents a solution based on using a modestly-priced little brown box. First, I present several experiments that demonstrate the lip sync problem. Next, I show the solution I implemented and several confirmation checks. After the demonstration I explain why the solution achieves its results. Watch the demonstration lipsync error video on YouTube here: http://www.youtube.com/watch?v=LTgC4J3UPgs

To conduct the experiments, I replay the same piece of video material from a digital video recorder (MySky HD) to the television (Samsung LED) and stereo amplifier system (Denon 500 PMA). A series of figures shows the equipment set-up for each demonstration. I located a home video camera in front of the TV to record what I saw and what I heard. The video image is not great! However, the video image is sufficient to observe the lip sync problem and the effectiveness of the solution.

Explanation

Why does the solution work? The modern digital TV requires around 50 - 200 milliseconds to process the digital data it receives from a signal source, and display the image on the LED screen. There are many reasons for this delay. (See LCD Input Lag / Video Delay measurement ) The traditional analogue TV appears not to have possessed this delay ‘feature’ to a level that is noticeable. Furthermore, the higher quality image, and/or larger size of modern TVs renders the problem more apparent. Consequently,  if you route the sound output from your signal source - such as a digital video recorder - to your hi-fi stereo amplifier system, you hear the sound BEFORE the digital TV has processed and presented the image. 200 milliseconds is ⅕ th of a second. That is a very small time period. However, the problem it caused was noticeable immediately to me when I upgraded from our 1980s vintage television earlier this year to a Samsung LED TV. You will see and hear the delay in the demonstration video, Experiments 1, and 2.

Fortunately, my Samsung digital TV provides an optical digital audio output that appears to be synchronized closely with the delayed TV image. Consequently, to solve the problem, you must take the digital audio signal output from the TV into your hi-fi stereo system. I credit the MySky technician who installed the MySky High Definition digital recorder for that solution strategy.

Experiment 3 shows how the audio signal broadcast from the TV is more closely synchronized with the video image … However, the sound quality is rather unacceptable since it is produced from small 2 x 10 Watt speakers in the TV, rather than my 2 x 130 Watt Denon hi-fi with KEF speakers! My hi-fi is a classic 1980s vintage Denon 500 PMA hi-fi amplifier. Consequently, the Denon amp accepts only analogue audio input, via RCA cables. Rather than purchase a new digital surround-sound hi-fi system, I wished to retain this vintage legacy system for several reasons. The Denon is not broken. It sound great. Furthermore, it provides a key component of the method I use to converting my vinyl LP collection into a digital format via a Zoom H4n Digital Recorder - but that’s another story.

To the rescue comes a little brown box that converts digital audio signals to analogue signals. In my case, I purchased an AC1603 Coaxial/Optical to R/L audio converter sourced via TradeMe, but ultimately supplied by Electus Distribution Pty Ltd, NSW Australia, http://www.electusdistribution.com.au. You also require a digital optical cable and an RCA Right and Left channel cable, male-male connectors. Connect the audio converter between the TV and your hi-fi system.  The video segment ‘Solution’ demonstrates the functioning of the digital-audio converter, and shows better synchronization between what you see (the lips) and hear. The ‘Check’ demonstration shows sound broadcast through both the TV and hi-fi. There is a much reduced, but acceptable delay between the two sound sources.  Image via Wikipedia

A comprehensive solution

Once you have this system installed, you should reroute your DVD and other audio-video signals first to the TV rather than through your hi-fi system. You must route BOTH the audio and video signals to the TV, so that the audio signal is relayed back to your hi-fi system. You will then achieve better synchronization of audio and video from these additional sources when you play the sound through your hi-fi system. This set-up enabled me to connect an AppleTV. This little black box connects via digital HDMI cable to the TV, and behaves like an additional audio-visual input. Now I can surf on my TV Youtube and Flickr from the comfort of my couch, listen in highest hi-fi to my iTunes music collection, and view in brilliant colour the photo albums stored on my computer.

The AppleTV is a pure digital device. It connects only to your TV via a HDMI cable. Without the setup I describe above, I would not have been able to use the AppleTV through my hi-fi system, which would defeat much of the purpose of the AppleTV box for relaying the iTunes collection to the hi-fi. Costs Coaxial/Optical Digital to R/L audio converter $NZ 84.50 including courier postage $5.00. Digital optical cable, 2 metre, $NZ 25 including courier postage $5.00 Total: $NZ 110 If you are not so fortunate with TradeMe, you can order a solution from Australia with total cost about $150. Try http://www.converters.tv/

Technical note

We assume that there is perfect synchronization between the video and audio image recorded on the video camera. We also assume that there are no losses in synchronization during the creation of the movie from iMovie, the upload and processing by YouTube, and your viewing of the video over the internet! The demonstration works satisfactorily on my computer! I may have been slightly hyperbolic about my estimate of my TV’s video processing delay I estimated 200 milliseconds. The point is, I notice the delay, and some readers/viewers will also notice the delay. Apparently, some people can notice a delay of 40 milliseconds. Various hobbyists and gamers measure and publish the delay and a report from Stanford University report on how the delay affects peoples’ perceptions of the message presented. See Reeves & Voelker (1993), LCD HDTV Input Lag Tests, and LCD Input Lag / Video Delay measurement, (below).

IMPORTANT NOTE ABOUT VIDEO PLAYBACK VIA THE INTERNET

I have viewed the lip-sync video on YouTube. If you watch it whilst downloading, you sometimes get lip-sync failure throughout the whole video! Wait for the video to download before playing it!

Equipment used

• Television: Samsung LED TV UA32.B6000, 32 inch(!), 2 x 10 Watt, 1920 x 1080 @ 60 Hz
• Digital recorder: MySky HDi
• HiFi amplifier: Denon PMA-500V 2 x 130 Watt
• Digital-analogue converter: AC1603 Coaxial/Optical to R/L audio converter
• Video camera: Panasonic 3CCD PV-GS150
• Microphone: Zoom H4n Handy Recorder

Further reading

Delay measurement and effects LCD Input Lag / Video Delay measurement (PC & console games responsiveness) - AVForums.com. (n.d.). Retrieved November 4, 2010, from http://www.avforums.com/forums/lcd-led-lcd-tvs/612503-lcd-input-lag-video-delay-measurement-pc-console-games-responsiveness.html

LCD HDTV Input Lag Tests. (2008, September 8). ARogan. Retrieved November 18, 2010, from http://blog.arogan.com/2008/09/lcd-hdtv-input-lag-tests.html Reeves, B., & Voelker, D. (1993, October).

Effects of Audio-Video Asynchrony on Viewer’s Memory, Evaluation of Content and Detection Ability - Research Report Prepared for 
Pixel Instruments. Stanford University. Retrieved 18 November, 2010, from http://www.lipfix.com/file/doc/stanfordresearch.pdf

More advanced solutions: Audio delay boxes In some cases the miss-match between audio and video signal originates in the source signals. For instance, DVD audio tracks may be out of sync with the video track. In these cases, more sophisticated solutions are available. An audio delay box can be used to match the audio and video signals whatever and whenever the source is transmitted. These are rather more expensive solutions. For example, see AD-100 Audio Delay Box. (n.d.). Retrieved November 4, 2010, from http://www.datavideo.us/products/tbcs-and-audio-delay/ad-100-audio-delay-box.html

Lip-Sync Error Technical Details. (n.d.). . Retrieved November 18, 2010, from http://www.lipfix.com/technical_details.html

Lip-Sync Error? Felston DD740. (n.d.). . Retrieved November 4, 2010, from http://www.lipfix.com/lip_sync_error.html

Lip Sync Error Fix. Digital Audio Delay - Plasma TV, LCD, DLP - Felston. (n.d.).  Retrieved November 4, 2010, from http://www.felston.com/ An intermediate-price solution uses a standard digital audio effects processor. Not quite so user-friendly to use as the Felston approach. Possible cheap lip sync solution (for non lip sync equipped AV amps) -

AVForums.com. (n.d.).  Retrieved November 4, 2010, from http://www.avforums.com/forums/av-amplifiers-receivers/379273-possible-cheap-lip-sync-solution-non-lip-sync-equipped-av-amps.html

BEHRINGER: FEX800. (n.d.). Retrieved November 4, 2010, from http://www.behringer.com/EN/Products/FEX800.aspx

Digital to analogue audio converters

HDTV, DVI, HDMI, VGA, PAL, NTSC, CGA, EGA, RGB Video Scan Converters Sales. (n.d.). Retrieved November 11, 2010, from http://www.converters.tv/

Digital to Analog Decoder, Converts digital Audio to analog audio, Gefen TV GTV-DD-2AA. (n.d.). Retrieved November 4, 2010, from http://www.ramelectronics.net/audio-video/audio-converters-extenders/analog-to-digital-digital-to-analog/digital-to-analog-decoder-gtv-dd-2-aa/prodGTVDD2AA.html

GefenTV+Digital+Audio++to+Analog+Adapter. (n.d.). . Retrieved November 4, 2010, from http://www.gefen.com/gefentv/gtvproduct.jsp?prod_id=5279

Related articles

• Perfect sounds: building the perfect hi-fi (telegraph.co.uk)
• Yamaha Goes Classic: Hi-Fi Components Combine Rock-Solid Vintage Design, Modern Sonic Purity (prweb.com)
• Denon Celebrates Its 100th Anniversary With Debut of Special Edition Product Collection (eon.businesswire.com)
• Flat-screen TV upgrade 2010: What a difference five years makes (news.cnet.com)
• How to have the music quality of 30 yrs ago with today’s music services (english.martinvarsavsky.net)
• The lip-sync problem in digital television: a demonstration and solution (mynstrel.blogspot.com)
• Hands-On With Apple’s AirPlay in iOS 4.2 (nytimes.com)
• Updates to bring AirPlay to Denon, Marantz A/V receivers (macworld.com)