EVALUATION OF SIMULTANEOUS IDENTITY, AGE AND GENDER RECOGNITION FOR CROWD FACE MONITORING
DOI:
https://doi.org/10.11113/aej.v13.17612Keywords:
Age Estimation, Crowd Monitoring, Deep Learning, Facial Recognition, Gender PredictionAbstract
Nowadays, facial recognition combined with age estimation and gender prediction has been deeply involved with the factors associated with crowd monitoring. This is considered to be a major and complex job for humans. This paper proposes a unified facial recognition system based on already available deep learning and machine learning models (i.e., FaceNet, ResNet, Support Vector Machine, AgeNet and GenderNet) that automatically and simultaneously performs person identification, age estimation and gender prediction. Then the system is evaluated on a newly proposed multi-face, realistic and challenging test dataset. The current face recognition technology primarily focuses on static datasets of known identities and does not focus on novel identities. This approach is not suitable for continuous crowd monitoring. In our proposed system, whenever novel identities are found during inference, the system will save those novel identities with an appropriate label for each unique identity and the system will be updated periodically in order to correctly recognise those identities in the future inference iterations. However, extracting the facial features of the whole dataset whenever a new identity is detected is not an efficient solution. To address this issue, we propose an incremental feature extraction based training method which aims to reduce the computational load of feature extraction. When tested on the proposed test dataset, our proposed system correctly recognizes pre-trained identities, estimates age, and predicts gender with an average accuracy of 49%, 66.5% and 93.54% respectively. We conclude that the evaluated pre-trained models can be sensitive and not robust to uncontrolled environment (e.g., abrupt lighting conditions).
References
He, K., Zhang, X., Ren, S., and Sun, J. 2016. Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 770-778. DOI: https://doi.org/10.1109/CVPR.2016.90
Schroff, F., Kalenichenko, D., and Philbin, J. 2015. FaceNet: A unified embedding for face recognition and clustering. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 815-823. DOI: https://doi.org/10.1109/CVPR.2015.7298682
Jonsson, K., Kittler, K., Li, Y.P., and Matas, J. 2002. Support vector machines for face authentication. Image and Vision Computing. 20 (5-6): 369-375. DOI: https://doi.org/10.1016/S0262-8856(02)00009-4
Levi, G., and Hassncer, T. 2015. Age and gender classification using convolutional neural networks. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. 34-42. DOI: https://doi.org/10.1109/CVPRW.2015.7301352
Kelly, M. D. 1973. Visual Identification of People by Computer. Department of Computer Science, Stanford University
Takeo, K. 1973. Picture Processing by Computer Complex and Recognition of Human Faces. Kyoto University, Ph. D. thesis edition, 1973.
Sirovich, L. and Kirby, M. 1987. Low-dimensional procedure for the characterization of human faces. Journal of the Optical Society of America A. 4(3): 519. DOI: https://doi.org/10.1364/JOSAA.4.000519
Price, J.R. and Gee, T.F. 2005. Face recognition using direct, weighted linear discriminant analysis and modular subspaces. Pattern Recognition. 38(2): 209-219. DOI: https://doi.org/10.1016/S0031-3203(04)00273-0
He, X., Yan, S., Hu, Y., Niyogi, P. & Zhang, J.H. 2005. Face recognition using Laplacianfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence. 27(3): 328-340. DOI: https://doi.org/10.1109/TPAMI.2005.55
Chen, d., Cao, X., Wang, L., Wen, F. & Sun, J. 2012. Bayesian face revisited: A joint formulation. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2012. 7574: 566-579. DOI: https://doi.org/10.1007/978-3-642-33712-3_41
Moghaddam, B., Wahid, W. and Pentland, A. 1998. Beyond eigenfaces: Probabilistic matching for face recognition. Proceedings - 3rd IEEE International Conference on Automatic Face and Gesture Recognition, FG 1998. 30-35.
Learned-Miller, E., Huang, G.B., RoyChowdhury, A., Li, H. and Hua, G. 2016. Labeled faces in the wild: A survey. Advances in face detection and facial image analysis, Springer. 189-248. DOI: https://doi.org/10.1007/978-3-319-25958-1_8
Pentland, A., Moghaddam, B. and Starner, T. 1994. View-based and modular eigenspaces for face recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 84-91. DOI: https://doi.org/10.1109/CVPR.1994.323814
Wiskott, L., Fellous, J.M., Krüger, N. and Von der Malsburg, C. 1997. Face recognition by elastic bunch graph matching. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 1296(7): 456-463. DOI: https://doi.org/10.1007/3-540-63460-6_150
Albiol, A., Monzo, D., Martin, A., Sastre, J. and Albiol, A. 2008. Face recognition using HOG-EBGM. Pattern Recognition Letters. 29(10): 1537-1543. DOI: https://doi.org/10.1016/j.patrec.2008.03.017
Lei, Z., Pietikainen, M. and Li, S. Z. 2014. Learning discriminant face descriptor. IEEE Transactions on Pattern Analysis and Machine Intelligence. 36(2): 289-302. DOI: https://doi.org/10.1109/TPAMI.2013.112
Liu, C. 2004. Gabor-based kernel PCA with fractional power polynomial models for face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence. 26(5): 572-581. DOI: https://doi.org/10.1109/TPAMI.2004.1273927
Guillaumin, M., Verbeek, J. and Schmid, C. 2009. Is that you? Metric learning approaches for face identification. Proceedings of the IEEE International Conference on Computer Vision. 498-505. DOI: https://doi.org/10.1109/ICCV.2009.5459197
Taigman, Y., Yang, M., Ranzato, M. and Wolf, L. 2014. DeepFace: Closing the gap to human-level performance in face verification. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1701-1708. DOI: https://doi.org/10.1109/CVPR.2014.220
Sun, Y., Wang, X. and Tang, X. 2014. Deep learning face representation from predicting 10,000 classes. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1891-1898. DOI: https://doi.org/10.1109/CVPR.2014.244
Deng, J., Guo, J., Xue, N. and Zafeiriou, S. 2019. ArcFace: Additive angular margin loss for deep face recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2019-June: 4685-4694. DOI: https://doi.org/10.1109/CVPR.2019.00482
Wan, W. and Lee, H. J. 2017. FaceNet Based Face Sketch Recognition. Proceedings - 2017 International Conference on Computational Science and Computational Intelligence, CSCI. 432-436
Jose, E., Greeshma, M., Mithun Haridas, T. P. and Supriya, M. H. 2019, March. Face Recognition based Surveillance System Using FaceNet and MTCNN on Jetson TX2. 2019 5th International Conference on Advanced Computing and Communication Systems, ICACCS. 608-613. DOI: https://doi.org/10.1109/ICACCS.2019.8728466
Nyein, T. and Oo, A.N. 2019, November. University Classroom Attendance System Using FaceNet and Support Vector Machine. 2019 International Conference on Advanced Information Technologies, ICAIT. 171-176. DOI: https://doi.org/10.1109/AITC.2019.8921316
Golla, M.R. and Sharma, P. 2019. Performance evaluation of facenet on low resolution face images. Communications in Computer and Information Science. 839: 317-325. DOI: https://doi.org/10.1007/978-981-13-2372-0_28
William, I., Ignatius, D. R., Moses Setiadi, Rachmawanto, E. H., Santoso, H. A., & Sari, C. A. 2019, October. Face Recognition using FaceNet (Survey, Performance Test, and Comparison). Proceedings of 2019 4th International Conference on Informatics and Computing, ICIC 2019. DOI: https://doi.org/10.1109/ICIC47613.2019.8985786
Frearson, M. and Nguyen, K. 2020, November. Adversarial Attack on Facial Recognition using Visible Light. DOI: https://doi.org/10.36227/techrxiv.13725634.v1
Zeng, W. et al. 2020, January. Person Recognition Based on FaceNet under Simulated Prosthetic Vision. Journal of Physics: Conference Series. 1437(1): 012012. DOI: https://doi.org/10.1088/1742-6596/1437/1/012012
Cahyono, F. Wirawan, W. and Fuad Rachmadi, R. 2020, September. Face recognition system using facenet algorithm for employee presence. 4th International Conference on Vocational Education and Training, ICOVET 2020. 57-62. DOI: https://doi.org/10.1109/ICOVET50258.2020.9229888
Moura, A. F. S., Pereira, S. S. L., Moreira, M. W. L. and Rodrigues, J. J. P. C. 2020, January. Video Monitoring System using Facial Recognition: A Facenet-based Approach. IEEE Global Communications Conference, GLOBECOM 2020 – Proceedings. 1-6. DOI: https://doi.org/10.1109/GLOBECOM42002.2020.9348216
Nair, S. P., Abhinav Reddy, K., Alluri, P. K. and Lalitha, S. 2021. Face recognition and tracking for security surveillance. Journal of Intelligent and Fuzzy Systems. 41(5): 5337-5345. DOI: https://doi.org/10.3233/JIFS-189856
Xu, J. 2021, February. A deep learning approach to building an intelligent video surveillance system. Multimedia Tools and Applications. 80(4): 5495-5515. DOI: https://doi.org/10.1007/s11042-020-09964-6
Preetha, S., Sheela, S. V. 2021, December. Security Monitoring System Using FaceNet for Wireless Sensor Network
Shirahatti, S. G. C, K. H. S, S and Bangari, S. R. 2021. Face Recognition System For Real Time Applications Using Svm Combined With Facenet And Mtcnn. International Journal of Electrical Engineering and Technology (IJEET). 12(6): 328-335.
Wu, C. and Zhang, Y. 2021, March. MTCNN and FACENET Based Access Control System for Face Detection and Recognition. Automatic Control and Computer Sciences. 55(1): 102-112. DOI: https://doi.org/10.3103/S0146411621010090
Adhinata, F. D., Rakhmadani, D. P. and Wijayanto, D. 2021, April. Fatigue Detection on Face Image Using FaceNet Algorithm and K-Nearest Neighbor Classifier. Journal of Information Systems Engineering and Business Intelligence 7(1): 22-30. DOI: https://doi.org/10.20473/jisebi.7.1.22-30
Özbulak, G., Aytar, Y. and Ekenel, H. K. 2016. How transferable are CNN-based features for age and gender classification?. Lecture Notes in Informatics (LNI), Proceedings - Series of the Gesellschaft fur Informatik (GI). P-260: 1-6. DOI: https://doi.org/10.1109/BIOSIG.2016.7736925
Kwon, Y. H. and Lobo, N. D. V. 1999. Age classification from facial images. Computer Vision and Image Understanding. 74(1): 1-21. https://doi.org/10.1006/cviu.1997.0549
Guo, G. and Mu, G. 2013. Joint estimation of age, gender and ethnicity: CCA vs. PLS. 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2013. 1-6. DOI: https://doi.org/10.1109/FG.2013.6553737
Guo, G. and Mu, G., Fu, Y., Dyer, C. and Huang, T. 2009. A study on automatic age estimation using a large database. Proceedings of the IEEE International Conference on Computer Vision. 1986-1991.
Han, H., Otto, C., Liu, X. and Jain, A. K. 2015. Demographic estimation from face images: Human vs. machine performance. IEEE Transactions on Pattern Analysis and Machine Intelligence. 37(6): 1148-1161. DOI: https://doi.org/10.1109/TPAMI.2014.2362759
Eidinger, E., Enbar, R. and Hassner, T. 2014. Age and gender estimation of unfiltered faces. IEEE Transactions on Information Forensics and Security. 9(12): 2170-2179. DOI: https://doi.org/10.1109/TIFS.2014.2359646
Chen, J. C., Kumar, A., Ranjan, R., Patel, V. M., Alavi, A. & Chellappa, R. 2016. A cascaded convolutional neural network for age estimation of unconstrained faces. 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems, BTAS 2016. 1-8. DOI: https://doi.org/10.1109/BTAS.2016.7791154
Huerta, I., Fernández, C., Segura, C., Hernando, J. & Prati, A. 2015. A deep analysis on age estimation. Pattern Recognition Letters. 68: 239-249. DOI: https://doi.org/10.1016/j.patrec.2015.06.006
Golomb, B. A., Lawrence, D. T. and Sejnowski, T. J. 1991. Sexnet: A neural network identifies sex from human faces. Advances in Neural Information Processing Systems 3. 1(July): 572-577
Shan, C. 2012. Learning local binary patterns for gender classification on real-world face images. Pattern Recognition Letters. 33(4): 431-437. DOI: https://doi.org/10.1016/j.patrec.2011.05.016
Van De Wolfshaar, J., Karaaba, M. F. and Wiering, M. A. 2015. Deep convolutional neural networks and support vector machines for gender recognition. Proceedings - 2015 IEEE Symposium Series on Computational Intelligence, SSCI 2015. 188-195. DOI: https://doi.org/10.1109/SSCI.2015.37
Yi, D., Lei, Z. and Li, S. Z. 2015. Age estimation by multi-scale convolutional network. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 9005: 144-158. DOI: https://doi.org/10.1007/978-3-319-16811-1_10
Rothe, R., Timofte, R. and Van Gool, L. 2018. Deep Expectation of Real and Apparent Age from a Single Image Without Facial Landmarks. International Journal of Computer Vision. 126: 144-157. DOI: https://doi.org/10.1007/s11263-016-0940-3
Mansanet, J., Albiol, A. and Paredes, R. 2016. Local Deep Neural Networks for gender recognition. Pattern Recognition Letters. 70: 80-86. DOI: https://doi.org/10.1016/j.patrec.2015.11.015
Agarwal, T., Andhale, M., Khule, A. and Borse, R. 2021. Age and Gender Classification Based on Deep Learning. Techno-Societal 2020. 425-437. DOI: https://doi.org/10.1007/978-3-030-69921-5_43
Adhinata, F. D. and Junaidi, A. 2022, January. Gender Classification on Video Using FaceNet Algorithm and Supervised Machine Learning. International Journal of Computing and Digital Systems. 11(1): 199-208. DOI: https://doi.org/10.12785/ijcds/110116
Agbo-Ajala, O. and Viriri, S. 2020. Deeply Learned Classifiers for Age and Gender Predictions of Unfiltered Faces. Scientific World Journal. DOI: https://doi.org/10.1155/2020/1289408
Rouhsedaghat, M., Wang, Y., Ge, X., Hu, S., You, S. and Kuo, C. C. J. 2021, January. FaceHop: A Light-Weight Low-Resolution Face Gender Classification Method. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 12668: 169-183. DOI: https://doi.org/10.1007/978-3-030-68793-9_12
Garain, A., Ray, B., Singh, P. K., Ahmadian, A., Senu, N. and Sarkar, R. 2021. GRA_Net: A Deep Learning Model for Classification of Age and Gender from Facial Images. IEEE Access. 9: 85672-85689. DOI: https://doi.org/10.1109/ACCESS.2021.3085971
Kharchevnikova A. S. and Savchenko, A. V. 2018, October. Neural Networks in Video-Based Age and Gender Recognition on Mobile Platforms. Optical Memory and Neural Networks (Information Optics). 27(4): 246-259. DOI: https://doi.org/10.3103/S1060992X18040021
Duan, M., Li, K., Ouyang, A., Win, K. N., Li, K. & Tian, Q. 2020, May. EGroupNet: A Feature-enhanced Network for Age Estimation with Novel Age Group Schemes. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM). 16(2). DOI: https://doi.org/10.1145/3379449
Savchenko, A. V. 2019. Efficient facial representations for age, gender and identity recognition in organizing photo albums using multi-output ConvNet. PeerJ Computer Science. 2019(6). DOI: https://doi.org/10.7717/peerj-cs.197
Hosseini, S., Lee, S. H., Kwon,H. J., Il Koo, H. and Cho, N. I. 2018, May. Age and gender classification using wide convolutional neural network and Gabor filter. 2018 International Workshop on Advanced Image Technology, IWAIT 2018. 1-3. DOI: https://doi.org/10.1109/IWAIT.2018.8369721
Khan, K., Attique, M., Syed, I., Sarwar, G., Irfan, M. A. and Khan, R. U. 2019, June. A Unified Framework for Head Pose, Age and Gender Classification through End-to-End Face Segmentation. Entropy 2019. 21(7): 647. DOI: https://doi.org/10.3390/e21070647
Duan, M., Li, K., Yang, C. and Li, K. 2018. A hybrid deep learning CNN-ELM for age and gender classification. Neurocomputing. 275: 448-461. DOI: https://doi.org/10.1016/j.neucom.2017.08.062
Benkaddour, M. K. 2021, August. CNN Based Features Extraction for Age Estimation and Gender Classification. Informatica. 45(5): 697-703. DOI: https://doi.org/10.31449/inf.v45i5.3262
Zhang, Z., Song, Y. and Qi, H. 2017, November. Age progression/regression by conditional adversarial autoencoder. Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. 4352-4360. DOI: https://doi.org/10.1109/CVPR.2017.463
