ROBUST HUMAN DETECTION WITH OCCLUSION HANDLING BY FUSION OF THERMAL AND DEPTH IMAGES FROM MOBILE ROBOT
DOI:
https://doi.org/10.11113/jt.v78.9271Keywords:
Human detection, occlusion handling, mobile robot, depth imaging, thermal imagingAbstract
In this paper, a robust surveillance system to enable robots to detect humans in indoor environments is proposed. The proposed method is based on fusing information from thermal and depth images which allows the detection of human even under occlusion. The proposed method consists of three stages; pre-processing, ROI generation and object classification. A new dataset was developed to evaluate the performance of the proposed method. The experimental results show that the proposed method is able to detect multiple humans under occlusions and illumination variations. Â
References
M. T. Ahmed and S. H. M. Amin, . 2015. Comparison of Face Recognition Algorithms for Human-Robot Interactions. J. Teknol. 2: 73–78.
T. Wilhelm, H.-J. Böhme, and H.-M. Gross, . 2004. A Multi-Modal System For Tracking And Analyzing Faces On A Mobile Robot. Rob. Auton. Syst. Aug. 2004. 48(1): 31–40.
R. C. Luo, A. C. Tsai, and C. T. Liao, . 2007. Face Detection and Tracking for Human Robot Interaction through Service Robot. The 33rd Annual Conference of the IEEE Industrial Electronics Society (IECON). 2818–2823.
W. R. Schwartz, R. Gopalan, R. Chellappa, and L. S. Davis,. 2009. Robust Human Detection Under Occlusion by Integrating Face and Person Detectors. Lect. Notes Comput. Sci. Int. Conf. Biometrics. 5558: 970–979.
M. Bennewitz, G. Cielniak, and S. Thrun. 2005. Learning Motion Patterns of People for Compliant Robot Motion. Int. J. Rob. Res. 24(3): 31–48.
A. Møgelmose, C. Bahnsen, T. B. Moeslund, A. Clapés, and S. Escalera, . Tri-modal Person Re-identification with RGB . Depth and Thermal Features.
W. Choi, C. Pantofaru, and S. Savarese, . 2011. Detecting and tracking people using an RGB-D camera via multiple detector fusion. Proc. IEEE Int. Conf. Comput. Vis. 1076–1083.
H. Zhang, C. Reardon, and L. E. Parker, . 2013. Real-time multiple human perception with color-depth cameras on a mobile robot. IEEE Trans. Cybern. 43(5): 1429–1441.
W. Choi, C. Pantofaru, and S. Savarese, . 2013. A general framework for tracking multiple people from a moving camera. IEEE Trans. Pattern Anal. Mach. Intell. 35(7): 1577–1591.
M. Correa, G. Hermosilla, R. Verschae, and J. Ruiz-del-Solar, . 2011. Human Detection and Identification by Robots Using Thermal and Visual Information in Domestic Environments. J. Intell. Robot. Syst. Jul. 2011. 66(1–2): 223–243.
A. Treptow, G. Cielniak, and T. Duckett, . 2006. Real-Time People Tracking For Mobile Robots Using Thermal Vision. Rob. Auton. Syst. 54: 729–739.
F. Guan, L. Y. Li, S. S. Ge, and A. P. Loh, . 2007. Robust Human Detection And Identification by using Stereo and Thermal Images In . Human 1 St Reading. 65.
L. Xia, C. Chen, and J. K. Aggarwal, . 2011. Human Detection Using Depth Information by Kinect. Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE. 15–22.
S. Ikemura and H. Fujiyoshi, . 2011. Real-Time Human Detection using Relational Depth Similarity Features. Computer Vision – ACCV 2010, Lecture Notes in Computer Science . 6495: 25–38.
G. Medioni, A. R. J. François, M. Siddiqui, K. Kim, and H. Yoon, . 2007. Robust real-time vision for a personal service robot. Comput. Vis. Image Underst. Oct. 2007. 108(1–2): 196–203.
H.-J. Böhme, T. Wilhelm, J. Key, C. Schauer, C. Schröter, H.-M. Groß, and T. Hempel, . 2003. An approach to multi-modal human–machine interaction for intelligent service robots. Rob. Auton. Syst. Jul. 2003. 44(1): 83–96.
J. Satake and J. Miura, . 2009. Robust Stereo-Based Person Detection and Tracking for a Person Following Robot. IEEE International Conference on Robotics and Automation. May.
L. Lit and W. Huangt, . 2004. Stereo-Based Human Detection For Mobile Service Robots. 8th Internasional Conference on Control, Automation, Robotics and Vision. December. 6–9.
U. Meis, M. Oberlander, and W. Ritter, . 2004. Reinforcing the Reliability of Pedestrian Detection in Far-infrared Sensing. IEEE Intelligent Vehicles Symposium. 779–783.
S. S. Mudaly, . Novel Computer-Based Infrared Pedestrian Data-Acquisition. Electronics Letters. 371–372.
H. Nanda and C. Park, . 2002. Probabilistic Template Based Pedestrian Detection in Infrared Videos. IEEE Intelligent Vehicles Symposium2002. 15–20.
M. Bertozzi, A. Member, A. Broggi, A. Fascioli, T. Graf, and M. Meinecke, . 2004. Pedestrian Detection for Driver Assistance Using Multiresolution Infrared Vision. IEEE Trans. Veh. Technol. 53(6): 1666–1678.
N. Bellotto, S. Member, H. Hu, and S. Member, . 2009. Multisensor-Based Human Detection and Tracking for Mobile Service Robots. IEEE Trans. Syst. ManCybern. - Part B Cybern. 39(1): 167–181.
F. Jurado, G. Palacios, F. Flores, and H. M. Becerra, . 2014. Vision-Based Trajectory Tracking System for an Emulated Quadrotor UAV. Asian J. Control. 16(3): 729–741.
D. Y. Gared and X. Ding, . Image Fusion for Concealed Weapon Detection. Int. J. Eng. Res. Technol. 2(2): 1–4.
T.-H. Chang, S. Gong, and E.-J. Ong, . 2000. Tracking Multiple People under Occlusion Using Multiple Cameras. Proc. 11th British Machine Vision Conference.
S. L. Dockstader and A. M. Tekalp,. 2001. Multiple camera fusion for multi-object tracking. Proceedings 2001 IEEE Workshop on Multi-Object Tracking. 95–102.
S. L. Dockstader and A. M. Tekalp, . 2001. Multiple Camera Tracking of Interacting and Occluded Human Motion. Proceedings of the IEEE . 89(10): 1441–1455.
R. Cucchiara, C. Grana, G. Tardini, R. Vezzani, and R. Emilia, . 2004. Probabilistic People Tracking for Occlusion Handling. Proceedings of the 17th International Conference on ICPR 2004. 132 – 135.
H. Eng, J. Wang, A. H. Kam, and W. Yau, . 2004. A Bayesian framework for robust human detection and occlusion handling using human shape model. Proceedings of the 17th International Conference on ICPR 2004. 2: 257–260.
Y. Wu, T. Yu, and G. Hua, . 2003. Tracking appearances with occlusions. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 789–795.
A. Senior, A. Hampapur, Y.-L. Tian, L. Brown, S. Pankanti, and R. Bolle, . 2006. Appearance models for occlusion handling. Image Vis. Comput. Nov. 2006. 24(11): 1233–1243.
H. T. Nguyen and A. W. M. Smeulders, . 2004. Fast occluded object tracking by a robust appearance filter. IEEE Trans. Pattern Anal. Mach. Intell. Aug. 2004. 26(8): 1099–104.
B. Wu and R. Nevatia, . 2005. Detection of Multiple Partially Occluded Humans in a Single Image by Bayesian Combination of Edgelet Part Detectorsin . Tenth IEEE International Conference on Computer Vision 2005. 1: 90 – 97 .
B. Wu and R. Nevatia, . 2006. Tracking of Multiple Partially Occluded Humans based on Static Body Part Detection. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1(0–7): .
B. Wu and R. Nevatia, . 2007. Detection and Tracking of Multiple Partially Occluded Humans by Bayesian Combination of Edgelet based Part Detectors. Int. J. Comput. Vis. 75(2): 247–266.
L. Wang, K. L. Chan, and G. Wang, . 2013. Human Detection with Occlusion Handling by Over-Segmentation and Clustering on Foreground Regions. The 11th Asian Conference on Computer Vision (ACCV2012). 197–208.
O. Camps and M. Sznaier, . 2001. Segmentation for Robust Tracking in the Presence of Severe Occlusion. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 483–489.
P. Viola and M. Jones, . 2001. Rapid object detection using a boosted cascade of simple features. Proc. 2001 IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognition. 1: 511-518.
H. S. Hadi, M. Rosbi, U. U. Sheikh, and S. H. M. Amin,. 2015. Fusion of Thermal and Depth Images for Occlusion Handling for Human Detection from Mobile Robot. The 10th Asian Control Conference 2015 (ASCC 2015).
Downloads
Published
Issue
Section
License
Copyright of articles that appear in Jurnal Teknologi belongs exclusively to Penerbit Universiti Teknologi Malaysia (Penerbit UTM Press). This copyright covers the rights to reproduce the article, including reprints, electronic reproductions, or any other reproductions of similar nature.
