Technique of video features extraction for audio-video speech recognition system

• Autorzy: Kubanek M.
• Języki publikacji: EN

Abstrakty

EN

Mainstream automatic speech recognition has focused almost exclusively on the acoustic signal. The performance of these systems degrades considerably in the real word in the presence of noise. It was needed novel approaches that use other orthogonal sources of information to the acoustic input that not only considerably improve the performance in severely degraded conditions, but also are independent to the type of noise and reverberation. Visual speech is one such source not perturbed by the acoustic environment and noise. In this paper, it was presented own approach to lip-tracking for audio-visual speech recognition system. It was presented video analysis of visual speech for extraction visual features from a talking person in color video sequences. It was developed a method for automatically face, eyes, lip's region, lip's corners and lip's contour de-tection. Finally, the paper will show results of lip-tracking depending on various factors (lighting, beard).

Słowa kluczowe

EN

speech recognition; audio-speech; visual speech; lip-tracking; face detection

PL

rozpoznawanie mowy; wykrywanie twarzy

• Wydawca: Wydawnictwo Politechniki Częstochowskiej
• Czasopismo: Computing, Multimedia and Intelligent Techniques
• Rocznik: 2005
• Tom: Vol. 1 nr 1
• Strony: 181--190
• Opis fizyczny: Bibliogr. 16 poz., rys., tab.

Twórcy

autor

Kubanek M.

• Częstochowa University of Technology, Institute of Computer and Information Sciences, ul. Dąbrowskiego 73, 42-200 Częstochowa, Poland

Bibliografia

• [1] Herda L., Fua P., Plankers R., Boulic R., Thalmann D., Skeleton-based motion capture for robust reconstruction of human motion, Proc. Computer Animation 2000, 77-83.
• [2] Jian Z., Kaynak M.N.N., Cheok A.D., Chung K.C., Real-time Lip-tracking For Virtual Lip Implementation in Virtual Environments and Computer Games, Proc. 2001 International Fuzzy Systems Conference 2001.
• [3] Aydin Y., Nakajama H., Realistic Articulated Character Positioning and Balance Control in Interactive Environments, Proc. Computer Animation 1999, 160-168.
• [4] Zhi Q., Kaynak M.N.N., Sengupta K., Cheok A.D., Ko C.C., A Study of the Modeling Aspects in Bimodal Speech Recognition, Proc. 2001 IEEE International Conference on Multimedia and Expo ICME 2001 200l.
• [5] Neti C., Potamianos G., Luttin J., Mattews I., Glotin H., Vergyri D., Sison J., Mashari A., Zhou J., Audio Visual Speech-Recognition, Workshop 2000 Final Report, October 12, 2000.
• [6] McGurk H., MacDonald J., Hearing lips and seeing voices, Nature 1976, 264, 746-748.
• [7] Massaro D.W., Stork D.G., Speech Recognition and Sensory Integration, American Scientist 1998, 86, 3, 236-244.
• [8] Hennecke M.E., Stork D.G., Prasad K.V., Visionary Speech: Looking Ahead to Practical Speechreading Systems, In Stork and Hennecke, 331-349.
• [9] Chan M.T., Zhang Y., Huang T.S., Real-time Lip-tracking and Bimodal Continous Speech Recognition, Proc. IEEE 2nd Workshop on Multimedia Signal Processing, Redondo Beach 1988, 65-70.
• [10] Steifelhagen R., Meier U., Yang J., Real-Time Lip-Tranking for Lipreading.
• [11] Kuchariev G., Kuzminski A., Techniki biometryczne, Cz. 1: Metody rozpoznawania twarzy, Wydział Informatyki, Politechnika Szczecińska 2003.
• [12] Gee A.H., Cipolla R., Fast Visual Tracking by Temporal Consensus, Technical Report CUED/F-INFENG/TR-207, University of Cambridge, February 1995.
• [13] Basu S., Oliver N., Pentland A., 3D Modeling and Tracking of Human Lip Motions, Proc. International Conference on Computer Vision 1998.
• [14] Kubanek K., Metoda krawędziowania EDGE do ekstrakcji cech obrazu ust w technice zintegrowanego rozpoznawania audio/video mowy, Informatyka Teoretyczna i Stosowana, Częstochowa 2003, 4, 115-125.
• [15] Kaucic R., Dalton B., Blake A., Real- Time Lip Tracking for Audio-Visual Speech Recognition Aplications, Proc. European Conf. Computer Vision, Cambridge 1996, 376-387.
• [16] Summerfield Q., MacLeod A., McGrath M., Broke M., Lips, Teeth and the Benefits of Lipreading, Handbook of Research on Face Processing, eds A.W. Young H.D. Ellis, Elsevier Science Publishers, 1989, 223-233.