×

A visual approach for driver inattention detection. (English) Zbl 1111.68637

Summary: Monitoring driver fatigue, inattention, and lack of sleep is very important in preventing motor vehicles accidents. A visual system for automatic driver vigilance has to address two fundamental problems. First of all, it has to analyze the sequence of images and detect if the driver has his eyes open or closed, and then it has to evaluate the temporal occurrence of eyes open to estimate the driver’s visual attention level. In this paper we propose a visual approach that solves both problems. A neural classifier is applied to recognize the eyes in the image, selecting two candidate regions that might contain the eyes by using iris geometrical information and symmetry. The novelty of this work is that the algorithm works on complex images without constraints on the background, skin color segmentation and so on. Several experiments were carried out on images of subjects with different eye colors, some of them wearing glasses, in different light conditions. Tests show robustness with respect to situations such as eyes partially occluded, head rotation and so on. In particular, when applied to images where people have eyes closed the proposed algorithm correctly reveals the absence of eyes. Next, the analysis of the eye occurrence in image sequences is carried out with a probabilistic model to recognize anomalous behaviors such as driver inattention or sleepiness. Image sequences acquired in the laboratory and while people were driving a car were used to test the driver behavior analysis and demonstrate the effectiveness of the whole approach.

MSC:

68T10 Pattern recognition, speech recognition
PDFBibTeX XMLCite
Full Text: DOI

References:

[1] K. Torkkola, N. Massey, C. Wood, Driver inattention detection through Intelligent Analysis of Readily Available Sensors, in: Proceedings of IEEE Conference on Intelligent Transportation Systems, Washington, DC, October 2004, pp. 326-331.; K. Torkkola, N. Massey, C. Wood, Driver inattention detection through Intelligent Analysis of Readily Available Sensors, in: Proceedings of IEEE Conference on Intelligent Transportation Systems, Washington, DC, October 2004, pp. 326-331.
[2] G. Yang, Y. Lin, P. Bhattacharya, A driver fatigue recognition model using fusion of multiple features, in: IEEE International Conference on Systems, Man and Cybernetics, Hawai USA, vol. 2, October 2005, pp. 1777-1784.; G. Yang, Y. Lin, P. Bhattacharya, A driver fatigue recognition model using fusion of multiple features, in: IEEE International Conference on Systems, Man and Cybernetics, Hawai USA, vol. 2, October 2005, pp. 1777-1784.
[3] S. Park, M. Trivedi, Driver activity analysis for intelligent vehicles: issues and development framework, in: Proceedings of IEEE Intelligent Vehicles Symposium, Las Vegas, USA, June 2005, pp. 795-800.; S. Park, M. Trivedi, Driver activity analysis for intelligent vehicles: issues and development framework, in: Proceedings of IEEE Intelligent Vehicles Symposium, Las Vegas, USA, June 2005, pp. 795-800.
[4] \( \langle;\) http://www.smarteye.se \(\rangle;\); \( \langle;\) http://www.smarteye.se \(\rangle;\)
[5] \( \langle;\) http://www.seeingmachine.com \(\rangle;\); \( \langle;\) http://www.seeingmachine.com \(\rangle;\)
[6] Y. Matsumoto, A. Zelinsky, An algorithm for real time stereo vision implementation of head pose and gaze direction measurement, in: Proceedings of the Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble France, March 2000, pp. 499-504.; Y. Matsumoto, A. Zelinsky, An algorithm for real time stereo vision implementation of head pose and gaze direction measurement, in: Proceedings of the Fourth International Conference on Automatic Face and Gesture Recognition, Grenoble France, March 2000, pp. 499-504.
[7] \( \langle;\) http://www.eyealert.com \(\rangle;\); \( \langle;\) http://www.eyealert.com \(\rangle;\)
[8] E. Wahlstrom, O. Masoud, N. Papanikolopoulos, Monitoring Driver Activities ITS Institute, Research Reports No. CTS 04-05, University of Minnesota.; E. Wahlstrom, O. Masoud, N. Papanikolopoulos, Monitoring Driver Activities ITS Institute, Research Reports No. CTS 04-05, University of Minnesota.
[9] Morimoto, C. H.; Koons, D.; Amir, A.; Flickner, M., Pupil detection and tracking using multiple light sources, Image Vision Comput., 18, 4, 331-335 (2000)
[10] Zhu, Z.; Ji, Q., Eye and gaze tracking for interactive graphic display, Mach. Vision Appl., 15, 3, 139-148 (2004)
[11] Zhu, Z.; Ji, Q.; Lan, P., Real time non intrusive monitoring and prediction of driver fatigue, IEEE Trans. Vehicular Technol., 53, 4, 1052-1068 (2004)
[12] Haro, A.; Flickner, M.; Essa, I., Detecting and tracking eyes by using their physiological properties, dynamics, and appearance, (Proceedings of IEEE CVPR 2000 (June 2000), Hilton Head Island: Hilton Head Island South Carolina), 163-168
[13] C. Cudalbu, B. Anastasiu, R. Radu, R. Cruceanu, E. Schmidt, E. Barth, Driver monitoring with a single high speed camera and IR illumination, in: International Symposium on Signals, Circuits and Systems, ISSCS 2005, Iasi Romania, vol. 1, July 2005, pp. 219-222.; C. Cudalbu, B. Anastasiu, R. Radu, R. Cruceanu, E. Schmidt, E. Barth, Driver monitoring with a single high speed camera and IR illumination, in: International Symposium on Signals, Circuits and Systems, ISSCS 2005, Iasi Romania, vol. 1, July 2005, pp. 219-222.
[14] Ji, Q.; Yang, X., Real time eye gaze and face pose tracking for monitoring driver vigilance, Real-Time Imaging, 8, 357-377 (2002) · Zbl 1010.68953
[15] E.M. Ayoob, R. Grace, A.M. Steinfeld, Driver-vehicle-interface (DVI) development of a drowsy driver detection and warning system for commercial vehicles, Technical Report CMU-RI-TR-05-46, Robotics Institute, Carnegie Mellon University, September 2005.; E.M. Ayoob, R. Grace, A.M. Steinfeld, Driver-vehicle-interface (DVI) development of a drowsy driver detection and warning system for commercial vehicles, Technical Report CMU-RI-TR-05-46, Robotics Institute, Carnegie Mellon University, September 2005.
[16] Rowley, H. A.; Baluja, S.; Kanade, T., Neural network-based face detection, IEEE Trans. Pattern Anal. Mach. Intell., 20, 1, 23-38 (1998)
[17] Hsu, R.; Mottleb, M.; Jain, A. K., Face detection in color images, IEEE Trans. Pattern Anal. Mach. Intell., 24, 5, 696-706 (2002)
[18] O. Jesorsky, K. Kirchberg, R. Frischholz, Robust face detection using the Hausdorff distance, in: Proceedings of the Third International Conference on Audio and Video-based Biometric Person Authentication, Lecture Notes in Computer Science, vol. 2091, Halmstad, Sweden, Springer, Berlin, 6-8 June 2001, pp. 90-95.; O. Jesorsky, K. Kirchberg, R. Frischholz, Robust face detection using the Hausdorff distance, in: Proceedings of the Third International Conference on Audio and Video-based Biometric Person Authentication, Lecture Notes in Computer Science, vol. 2091, Halmstad, Sweden, Springer, Berlin, 6-8 June 2001, pp. 90-95.
[19] Yang, M. H.; Kriegman, D.; Ahuja, N., Detecting faces in images: a survey, IEEE Trans. Pattern Anal. Mach. Intell., 24, 1, 34-58 (2002)
[20] Kawaguchi, T.; Rizon, M., Iris detection using intensity and edge information, Pattern Recognition, 36, 2, 549-562 (2003)
[21] Li, Y.; Qi, X.-l.; Wang, Y.-j., Eye detection by using fuzzy template matching and feature-parameter based judgement, Pattern Recognition Lett., 22, 10, 1111-1124 (2001) · Zbl 1013.68186
[22] Baskan, S.; Bulut, M.; Atalay, V., Projection based method for segmentation of human face and its evaluation, Pattern Recognition Lett., 23, 14, 1623-1629 (2002) · Zbl 1007.68924
[23] Y. Ma, X. Ding, Z. Wang, N. Wang, Robust Precise eye location under probabilistic frame work, in: Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, Seoul Korea, May 2004, pp. 339-344.; Y. Ma, X. Ding, Z. Wang, N. Wang, Robust Precise eye location under probabilistic frame work, in: Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, Seoul Korea, May 2004, pp. 339-344.
[24] H. Kashima, H. Hongo, K. Kato, K. Yamamoto, A robust iris detection method of facial and eye movement, in: Proceedings of VI 2001 Vision Interface Annual Conference Ottawa, Canada, 7-9 June 2001, pp. 9-14.; H. Kashima, H. Hongo, K. Kato, K. Yamamoto, A robust iris detection method of facial and eye movement, in: Proceedings of VI 2001 Vision Interface Annual Conference Ottawa, Canada, 7-9 June 2001, pp. 9-14.
[25] Sirohey, S.; Rosenfiled, A., Eye detection in a face image using linear and nonlinear filters, Pattern Recognition, 34, 7, 1367-1391 (2001) · Zbl 0978.68119
[26] Sirohey, S.; Rosenfiled, A.; Duric, Z., A method of detection and tracking iris and eyelids in video, Pattern Recognition, 35, 6, 1389-1401 (2002) · Zbl 1002.68795
[27] Smith, P.; Shah, M.; da Vitoria Lobo, N., Determining driver visual attention with one camera, IEEE Trans. Intell. Transp. Syst., 4, 4, 205-218 (2003)
[28] Feng, G. C.; Yuen, P. C., Multi cues eye detection on gray intensity image, Pattern Recognition, 34, 5, 1033-1046 (2001) · Zbl 1006.68855
[29] Chiang, C. C.; Tai, W. K.; Yang, M. T.; Huang, Y. T.; Jaung, C., A novel method for detecting lips, eyes and faces in real time, Real Time Imaging, 9, 4, 277-287 (2003)
[30] P. Wang, M.B. Green, Q. Ji, J. Wayman, Automatic eye detection and its validation, in: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, USA, vol. 3, June 2005, pp. 164-164.; P. Wang, M.B. Green, Q. Ji, J. Wayman, Automatic eye detection and its validation, in: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, USA, vol. 3, June 2005, pp. 164-164.
[31] P. Wang, Q. Ji, Learning Discriminant features for multiview face and eye detection automatic eye detection and its vadidation, in: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, USA, vol. 1, June 2005, pp. 373-379.; P. Wang, Q. Ji, Learning Discriminant features for multiview face and eye detection automatic eye detection and its vadidation, in: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, USA, vol. 1, June 2005, pp. 373-379.
[32] V.L. Neale, T. Dingus, S. Klauer, J. Sudweeks, M. Goodman, An overview of the 100-car naturalistic study and findings, in: Proceedings of 19th International Conference on Enhanced Safety of Vehicles, Washington, DC, June 6-9, 2005.; V.L. Neale, T. Dingus, S. Klauer, J. Sudweeks, M. Goodman, An overview of the 100-car naturalistic study and findings, in: Proceedings of 19th International Conference on Enhanced Safety of Vehicles, Washington, DC, June 6-9, 2005.
[33] T. Von Jan,T. Karnahl, K. Seifert, J. Hilgenstock, R. Zobel, Don’t sleep and drive-VW’s fatigue detection technology, in: Proceedings of 19th International Conference on Enhanced Safety of Vehicles, Washington, DC, June 6-9, 2005.; T. Von Jan,T. Karnahl, K. Seifert, J. Hilgenstock, R. Zobel, Don’t sleep and drive-VW’s fatigue detection technology, in: Proceedings of 19th International Conference on Enhanced Safety of Vehicles, Washington, DC, June 6-9, 2005.
[34] D.L. Smith, J. Chang, D. Cohen, J. Foley, R. Glasco, A simulation approach for evaluating the relative safety impact of driver distraction during secondary tasks, in: Proceedings of 12th World Congress on ITS, November 6-10, 2005, San Fancisco.; D.L. Smith, J. Chang, D. Cohen, J. Foley, R. Glasco, A simulation approach for evaluating the relative safety impact of driver distraction during secondary tasks, in: Proceedings of 12th World Congress on ITS, November 6-10, 2005, San Fancisco.
[35] P.S. Rau, Drowsy driver detection and warning system for commercial vehicle drivers: field operational test design, data analyses, and progress, in: Proceedings of 19th International Conference on Enhanced Safety of Vehicles, Washington, DC, June 6-9, 2005.; P.S. Rau, Drowsy driver detection and warning system for commercial vehicle drivers: field operational test design, data analyses, and progress, in: Proceedings of 19th International Conference on Enhanced Safety of Vehicles, Washington, DC, June 6-9, 2005.
[36] D’Orazio, T.; Guaragnella, C.; Leo, M.; Distante, A., A new algorithm for ball recognition using circle Hough transform and neural classifier, Pattern Recognition, 37, 393-408 (2004)
[37] Everitt, B. S.; Hand, D. J., Finite Mixture Distributions (1981), Chapman & Hall: Chapman & Hall New York · Zbl 0466.62018
[38] Dempster, A. P.; Laird, N. M.; Rubin, D. B., Maximum likelihood from incomplete data via the EM algorithm, J. R. Stat. Soc. Ser. B, 39, 1, 1-38 (1977) · Zbl 0364.62022
[39] J. Bilmes, A gentle tutorial on the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models, Technical Report ICSI-TR-97-021, International Computer Science Institute (ICSI), Berkeley, CA, 1997.; J. Bilmes, A gentle tutorial on the EM algorithm and its application to parameter estimation for Gaussian mixture and hidden Markov models, Technical Report ICSI-TR-97-021, International Computer Science Institute (ICSI), Berkeley, CA, 1997.
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.