Generating and Improving a Dataset of Masked Faces Using Data Augmentation

Waleed Ayad; Siraj Qays; Ali Al-Naji

doi:10.51173/jt.v5i2.1140

Authors

Waleed Ayad Electrical Engineering Technical College, Middle Technical University, Baghdad, Iraq.
Siraj Qays Electrical Engineering Technical College, Middle Technical University, Baghdad, Iraq.
Ali Al-Naji School of Engineering, University of South Australia, Mawson Lakes, SA 5095, Australia

DOI:

https://doi.org/10.51173/jt.v5i2.1140

Keywords:

COVID-19, Masked Face Dataset, Data Augmentation, Face Recognition

Abstract

Before the spread of the COVID-19 virus in 2020, modern face recognition systems performed excellently, but then the wearing of masks was imposed by countries on their population, which led to a noteworthy decrease in the discriminatory ability of those systems, where they had been trained on large-scale datasets of unmasked faces and not available large-scale masked faces datasets that time. To contribute to addressing the shortage of large-scale data sets that consist of people wearing masks, a developed method has been presented to create simulated masks and overlay them on faces in two main steps. The first step was to detect, align and crop the faces of unmasked faces datasets in a dataset and then apply simulated masks on the faces utilizing the dlib-ml library. This method was used to generate a dataset for masked faces (CASIA-mask). The second step used five techniques of data augmentation with the generated dataset. To evaluate the masked dataset and data augmentation, an accuracy of 96.4% was achieved by training one of the latest and most important facial recognition systems, FaceNet, on the masked dataset. The same system also achieved excellent results of 97.71% when trained on CASIA-mask and data augmentation together.

Downloads

Download data is not yet available.

Author Biography

Ali Al-Naji, School of Engineering, University of South Australia, Mawson Lakes, SA 5095, Australia

References

F. Schroff, D. Kalenichenko, and J. Philbin, “FaceNet: A unified embedding for face recognition and clustering,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 07-12-June, pp. 815–823, 2015.

W. Liu, Y. Wen, Z. Yu, M. Li, B. Raj, and L. Song, “SphereFace: Deep Hypersphere Embedding for Face Recognition,” Proc. IEEE Conf. on computer vision and pattern recognition, vol. 47, no. 5, pp. 212–220, 2017.

O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep Face Recognition,” Deep Learning in Visual Computing, 2015.

J. Deng, J. Guo, N. Xue, and S. Zafeiriou, “ArcFace: Additive angular margin loss for deep face recognition,” IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 1, pp. 4685–4694, June 2019.

Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “DeepFace: Closing the gap to human-level performance in face verification,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 1701–1708, 2014, doi: 10.1109/CVPR.2014.220.

G. B. Huang, M. Mattar, T. Berg, E. L. Labeled, R. Images, and E. Learned-miller, "Labeled faces in the wild: A database for studying face recognition in unconstrained environments," work. faces in'Real-Life'Images Detect. alignment, Recognit., 2008.

D. Yi, Z. Lei, S. Liao, and S. Z. Li, “Learning Face Representation from Scratch,” arXiv Prepr. arXiv preprint arXiv:1411.7923, Nov 2014.

Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep learning face attributes in the wild,” IEEE Int. Conf. on Computer Vision, vol. 2015 Inter, pp. 3730–3738, 2015.

Z. Zhu, G. Huang, J. Deng, Y. Ye, J. Huang, X. Chen, J. Zhu, T. Yang, J. Lu, D. Du, J. Zhou, “WebFace260M: A Benchmark Unveiling the Power of Million-Scale Deep Face Recognition,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 10487–10497, 2021.

A. Anwar and A. Raychowdhury, “Masked Face Recognition for Secure Authentication,” arXiv Prepr. arXiv2008.11104, pp. 1–8, August 2020.

J. Deng, J. Guo, X. An, Z. Zhu, and S. Zafeiriou, “Masked Face Recognition Challenge: The InsightFace Track Report,” Proc. IEEE Int. Conf. Comput. Vis., pp. 1437–1444, October 2021.

Z. Wang, G. Wang, B. Huang, Z. Xiong, Q. Hong, Wu, H., P. Yi, K. Jiang, N. Wang, Y. Pei, and H. Chen, “Masked Face Recognition Dataset and Application,” arXiv Prepr. arXiv2003.09093, pp. 1–3, March 2020.

Z. Zhu, G. Huang, J. Deng, Y. Ye, J. Huang, X. Chen, J. Zhu, T. Yang, J. Guo, J. Lu, and D. Du, “Masked Face Recognition Challenge: The WebFace260M Track Report,” arXiv Prepr. arXiv2108.07189, August 2021.

T. Mare et al., “A realistic approach to generate masked faces applied on two novel masked face recognition data sets,” arXiv Prepr. arXiv2109.01745, October 2021.

M. Asif and D. Kamthania, “Face Detection and Recognition with Mask,” IEEE Int. Conf. Autom. Logist. ICAL, vol. 10, pp. 121–126, October 2021.

D. E. King, “Dlib-ml: A machine learning toolkit,” J. Mach. Learn. Res., vol. 10, pp. 1755–1758, 2009.

B. Huang, Huang, B., Z. Wang, G. Wang, K. Jiang, K. Zeng, Z. Han, X. Tian, and Y. Yang, “When face recognition meets occlusion: A new benchmark,” ICASSP, IEEE Int. Conf. Acoust. Speech Signal Process. - Proc., vol. 1, pp. 1–5, Mar 2021.

F. Ding, P. Peng, Y. Huang, M. Geng, and Y. Tian, “Masked Face Recognition with Latent Part Detection,” 28th ACM Int. Conf. Multimed, pp. 2281–2289, October 2020.

K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, “Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks,” IEEE Signal Process. Lett., vol. 23, pp. 1499–1503, 2016.

T. H. Huang, K. Y. Cheng, and Y. Y. Chuang, “A collaborative benchmark for region of interest detection algorithms,” 2009 IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2009, pp. 296–303, 2009.

Y. Liu, Z. G. Li, and Y. C. Soh, “Region-of-interest based resource allocation for conversational video communication of H.264/AVC,” IEEE Trans. Circuits Syst. Video Technol., vol. 18, pp. 134–139, 2008.

H. Mliki, S. Dammak, and E. Fendri, “An improved multi-scale face detection using convolutional neural network,” Signal, Image Video Process., vol. 14, pp. 1345–1353, 2020.

W. Sun, H. Zhao, and Z. Jin, “A visual attention based ROI detection method for facial expression recognition,” Neurocomputing, vol. 296, pp. 12–22, 2018.

X. Wang, K. Wang, and S. Lian, “A Survey on Face Data Augmentation,” arXiv:1904.11685v1 [cs.CV] 26 Apr 2019.

Z. Pei, H. Xu, Y. Zhang, M. Guo, and Y. Yee-Hong, “Face recognition via deep learning using data augmentation based on orthogonal experiments,” Electron., vol. 8, no. 10, pp. 1–16, 2019, doi: 10.3390/electronics8101088.

H. Ben Fredj, S. Bouguezzi, and C. Souani, “Face recognition in unconstrained environment with CNN,” Vis. Comput., vol. 37, no. 2, pp. 217–226, 2021, doi: 10.1007/s00371-020-01794-9.

V. Uchoa, K. Aires, R. Veras, A. Paiva, and L. Britto, “Data Augmentation for Face Recognition with CNN Transfer Learning,” Int. Conf. Syst. Signals, Image Process., vol. 2020-July, pp. 143–148, 2020, doi: 10.1109/IWSSIP48289.2020.9145453.

M. Wang and W. Deng, “Deep face recognition: A survey,” Neurocomputing, vol. 429, pp. 215–244, 2021, doi: 10.1016/j.neucom.2020.10.081.

X. Peng, Z. Tang, F. Yang, R. S. Feris, and D. Metaxas, “Jointly Optimize Data Augmentation and Network Training: Adversarial Data Augmentation in Human Pose Estimation,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 2226–2234, 2018, doi: 10.1109/CVPR.2018.00237.

J. H. Lee, M. Z. Zaheer, M. Astrid, and S. I. Lee, “SmoothMix: A simple yet effective data augmentation to train robust classifiers,” IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Work., vol. 2020-June, pp. 3264–3274, 2020, doi: 10.1109/CVPRW50498.2020.00386.

M. Längkvist, L. Karlsson, and A. Loutfi, “Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning,” Pattern Recognit. Lett., vol. 42, pp. 11–24, 2014.