Computer Vision System for Facial Palsy Detection

Ali Saber Amsalam; Ali  Al-Naji; Ammar  Yahya Daeef; Javaan Chahl

doi:10.51173/jt.v5i1.1133

Authors

Ali Saber Amsalam Electrical Engineering Technical College, Middle Technical University, Baghdad, Iraq.
Ali Al-Naji School of Engineering, University of South Australia, Adelaide, Australia
Ammar Yahya Daeef Technical Institute for Administration, Middle Technical University, Baghdad, Iraq
Javaan Chahl School of Engineering, University of South Australia, Mawson Lakes, SA 5095, Australia

DOI:

https://doi.org/10.51173/jt.v5i1.1133

Keywords:

Non-Contact Palsy Detection, Computer Vision, Deep Learning, Digital Camera

Abstract

Facial palsy (FP) is a disorder that affects the seventh facial nerve, which makes the patient unable to control facial movements and expressions with other vital activities. It affects one side of the face, and it is usually diagnosed by the asymmetry of the two sides of the face through visual inspection by a doctor. However, the visual inspection is human-based, which is prone to errors because the doctor is exposed to omission due to fatigue and work stress. Therefore, it is important to develop a new method for detecting FP through artificial intelligence and use a more accurate computerized system to reduce the effort and cost of patients and increase the accuracy of diagnosis. This work aims to establish a safe, useful and high-accuracy diagnostic system for FP that can be used by the patient and proposes to detect FP using a digital camera and deep learning techniques automatically. The system could be used by the patient himself at home without needing to visit the hospital. The proposed system trained 570 images, including 200 images of FP palsy. The proposed FP system achieved an accuracy of 98%. This confirms the effectiveness of the proposed system and makes it an efficient medical examination tool for detecting FP.

Downloads

Download data is not yet available.

Author Biographies

Ali Al-Naji, School of Engineering, University of South Australia, Adelaide, Australia

Engineering in Medical Instrumentation Techniques

https://orcid.org/0000-0002-8840-9235

Javaan Chahl , School of Engineering, University of South Australia, Mawson Lakes, SA 5095, Australia

Professor of Sensor Systems (School of Engineering)

https://orcid.org/0000-0001-6496-0543

References

Barbosa, J., et al., Efficient quantitative assessment of facial paralysis using iris segmentation and active contour-based key points detection with hybrid classifier. BMC medical imaging, 2016. 16(1): p. 1-18.

Szczepura, A., et al., Raising the digital profile of facial palsy: National surveys of patients’ and clinicians’ experiences of changing UK treatment pathways and views on the future role of digital technology. Journal of medical Internet research, 2020. 22(10): p. e20406.

Baugh, R.F., et al., Clinical practice guideline: Bell’s palsy. Otolaryngology–Head and Neck Surgery, 2013. 149(3_suppl): p. S1-S27.

Ahmed, A., When is facial paralysis Bell palsy? Current diagnosis and treatment. Cleve Clin J Med, 2005. 72(5): p. 398-401.

O'Shea, K. and R. Nash, An introduction to convolutional neural networks. arXiv preprint arXiv:1511.08458, 2015.

Lindeberg, T., Scale invariant feature transform. 2012.

Dalal, N. and B. Triggs. Histograms of oriented gradients for human detection. in 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05). 2005. Ieee.

Li, Z., et al., A survey of convolutional neural networks: analysis, applications, and prospects. IEEE transactions on neural networks and learning systems, 2021.

Gu, J., et al., Recent advances in convolutional neural networks. Pattern recognition, 2018. 77: p. 354-377.

O’Mahony, N., et al. Deep learning vs. traditional computer vision. in Science and information conference. 2019. Springer.

Zou, W.W. and P.C. Yuen, Very low resolution face recognition problem. IEEE Transactions on image processing, 2011. 21(1): p. 327-340.

Hassaballah, M. and S. Aly, Face recognition: challenges, achievements and future directions. IET Computer Vision, 2015. 9(4): p. 614-626.

Song, A., et al., Neurologist standard classification of facial nerve paralysis with deep neural networks. Future Internet, 2018. 10(11): p. 111.

Abayomi-Alli, O.O., et al., Few-shot learning with a novel Voronoi tessellation-based image augmentation method for facial palsy detection. Electronics, 2021. 10(8): p. 978.

Parra-Dominguez, G.S., R.E. Sanchez-Yanez, and C.H. Garcia-Capulin, Facial paralysis detection on images using key point analysis. Applied Sciences, 2021. 11(5): p. 2435.

Nguyen, D.-P., M.-C. Ho Ba Tho, and T.-T. Dao, Enhanced facial expression recognition using 3D point sets and geometric deep learning. Medical & Biological Engineering & Computing, 2021. 59(6): p. 1235-1244.

Chiesa-Estomba, C.M., et al., Machine learning models for predicting facial nerve palsy in parotid gland surgery for benign tumors. journal of surgical research, 2021. 262: p. 57-64.

Alagha, M.A., et al., Objective grading facial paralysis severity using a dynamic 3D stereo photogrammetry imaging system. Optics and Lasers in Engineering, 2022. 150: p. 106876.

Guo, Z., et al., Facial imaging and landmark detection technique for objective assessment of unilateral peripheral facial paralysis. Enterprise Information Systems, 2021: p. 1-17.

Liu, Y., et al. Automatic Assessment of Facial Paralysis Based on Facial Landmarks. in 2021 IEEE 2nd International Conference on Pattern Recognition and Machine Learning (PRML). 2021. IEEE.

Parra-Dominguez, G.S., C.H. Garcia-Capulin, and R.E. Sanchez-Yanez, Automatic Facial Palsy Diagnosis as a Classification Problem Using Regional Information Extracted from a Photograph. Diagnostics, 2022. 12(7): p. 1528.

Xu, C., et al., A novel facial emotion recognition method for stress inference of facial nerve paralysis patients. Expert Systems with Applications, 2022. 197: p. 116705.

Azuma, T., et al., New method to evaluate sequelae of static facial asymmetry in patients with facial palsy using three-dimensional scanning analysis. Auris Nasus Larynx, 2022.

Vletter, C., et al., Towards an Automatic Diagnosis of Peripheral and Central Palsy Using Machine Learning on Facial Features. arXiv preprint arXiv:2201.11852, 2022.

Zahara, L., et al. The facial emotion recognition (FER-2013) dataset for prediction system of micro-expressions face using the convolutional neural network (CNN) algorithm based Raspberry Pi. in 2020 Fifth international conference on informatics and computing (ICIC). 2020. IEEE.

Khaireddin, Y. and Z. Chen, Facial emotion recognition: State of the art performance on FER2013. arXiv preprint arXiv:2105.03588, 2021.

Arriaga, O., M. Valdenegro-Toro, and P. Plöger, Real-time convolutional neural networks for emotion and gender classification. arXiv preprint arXiv:1710.07557, 2017.

Hsu, G.-S.J., J.-H. Kang, and W.-F. Huang, Deep hierarchical network with line segment learning for quantitative analysis of facial palsy. IEEE Access, 2018. 7: p. 4833-4842.

Pasquale, C., et al. Recovery from Idiopathic Facial Paralysis (Bell’s Palsy) Using Photobiomodulation in Patients Non-Responsive to Standard Treatment: A Case Series Study. in Photonics. 2021. MDPI.

Haase, D., et al., Automated and objective action coding of facial expressions in patients with acute facial palsy. European Archives of Oto-Rhino-Laryngology, 2015. 272(5): p. 1259-1267.

Chandaliya, P.K., et al. Scdae: Ethnicity and gender alteration on clf and utkface dataset. in International Conference on Computer Vision and Image Processing. 2019. Springer.

Barbosa, J., W.-K. Seo, and J. Kang, paraFaceTest: an ensemble of regression tree-based facial features extraction for efficient facial paralysis classification. BMC Medical Imaging, 2019. 19(1): p. 1-14.

Wang, T., et al., Automatic facial paralysis evaluation augmented by a cascaded encoder network structure. IEEE Access, 2019. 7: p. 135621-135631.

Grandini, M., E. Bagli, and G. Visani, Metrics for multi-class classification: an overview. arXiv preprint arXiv:2008.05756, 2020.