A REVIEW ON AUGMENTED REALITY TRACKING METHODS FOR MAINTENANCE OF ROBOTS

Authors

  • Ye Sheng Koh School of Electrical Engineering, Faculty of Engineering, Universiti Teknologi Malaysia, 81310, UTM Johor Bahru, Johor, Malaysia https://orcid.org/0000-0002-6349-8454
  • Kai Woon Goh School of Electrical Engineering, Faculty of Engineering, Universiti Teknologi Malaysia, 81310, UTM Johor Bahru, Johor, Malaysia
  • Marvin Dares School of Electrical Engineering, Faculty of Engineering, Universiti Teknologi Malaysia, 81310, UTM Johor Bahru, Johor, Malaysia
  • Che Fai Yeong School of Electrical Engineering, Faculty of Engineering, Universiti Teknologi Malaysia, 81310, UTM Johor Bahru, Johor, Malaysia
  • Eileen Su Lee Ming School of Electrical Engineering, Faculty of Engineering, Universiti Teknologi Malaysia, 81310, UTM Johor Bahru, Johor, Malaysia
  • Mohd. Shahrizal Sunar bMedia and Game Innovation Centre of Excellence, Institute of Human Centered Engineering, Universiti Teknologi Malaysia, 81310 UTM Johor Bahru, Johor, MalaysiacSchool of Computing, Faculty of Engineering, Universiti Teknologi Malaysia, 81310 UTM Johor Bahru, Johor, Malaysia
  • Yik Shin Tey DF Automation and Robotics Sdn. Bhd., 81310, Johor Bahru, Johor, Malaysia

DOI:

https://doi.org/10.11113/jurnalteknologi.v83.14907

Keywords:

Augmented reality, AR, maintenance, robot, tracking

Abstract

Augmented reality (AR) in maintenance is a broad subject with many nuances when it comes to their implementation. The applications of these systems range from maintenance of large-scale assets such as buildings to smaller scale assets such as robots. Applications of AR in maintenance typically serves as a visual guide to assist users in diagnosis or steps needed to be performed for maintenance. In this paper, the tracking methods utilized in AR-based maintenance for robots are qualitatively evaluated. The reviewed works in this paper are between the years of 2015 to 2020 to ensure that the AR tracking methods are relatively state of the art. It is found that applications of AR-based maintenance for robots are uncommon in the scope defined in this research especially in the industrial environment as most reviewed works are conducted in a laboratory setting. In addition to that, it is found that marker-based tracking methods are commonly utilized in these applications.

References

Amin, D. and S. Govilkar, Feb. 2015, “Comparative Study of Augmented Reality Sdk’s,†Int. J. Comput. Sci. Appl., vol. 5, no. 1, pp. 11–26.

DOI: https://doi.org/10.5121/ijcsa.2015.5102

Mekni, M. and A. Lemieux, “Augmented Reality : Applications , Challenges and Future Trends,†2014.

Schmalstieg, D. and T. Hollerer, “Augmented reality: Principles and practice,†in 2017 IEEE Virtual Reality (VR), 2017, pp. 425–426.

DOI: https://doi.org/10.1109/VR.2017.7892358

Palmarini, R., J. A. Erkoyuncu, R. Roy, and H. Torabmostaedi, Feb. 2018, “A systematic review of augmented reality applications in maintenance,†Robot. Comput. Integr. Manuf., vol. 49, pp. 215–228

DOI: https://doi.org/10.1016/j.rcim.2017.06.002

Rongting, Z., S. Yiran, H. Tongliang, and F. Asmi, “Applying Augmented Reality Technology to E-Learning: Science Educational AR Products as an Example,†in 2016 IEEE 13th International Conference on e-Business Engineering (ICEBE), Nov. 2016, pp. 129–133.

DOI: https://doi.org/10.1109/ICEBE.2016.030

Detmer, F. J., J. Hettig, D. Schindele, M. Schostak, and C. Hansen, 2017, “Virtual and Augmented Reality Systems for Renal Interventions: A Systematic Review,†IEEE Rev. Biomed. Eng., vol. 10, pp. 78–94.

DOI: https://doi.org/10.1109/RBME.2017.2749527

Alrashed, M., Y. Yadekar, J. A. Erkoyuncu, and Y. Zhao, Jan. 2016, “Examination of Robotic Aerospace Engines Maintenance Supported by Augmented Reality through Cloud Manufacturing,†Proc. Int. Conf. Artif. Life Robot., vol. 21, pp. 445–449.

DOI: https://doi.org/10.5954/ICAROB.2016.GS3-1

Luxenburger, A. et al., “Augmented Reality for Human-Robot Cooperation in Aircraft Assembly,†in 2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), Dec. 2019, pp. 263–2633.

DOI: https://doi.org/10.1109/AIVR46125.2019.00061

Flatt, H., N. Koch, A. Günter, C. Röcker, and J. Jasperneite, A Context-Aware Assistance System for Maintenance Applications in Smart Factories based on Augmented Reality and Indoor Localization. 2015.

DOI: https://doi.org/10.1109/ETFA.2015.7301586

Brizzi, F., L. Peppoloni, A. Graziano, E. D. Stefano, C. A. Avizzano, and E. Ruffaldi, 2018, “Effects of Augmented Reality on the Performance of Teleoperated Industrial Assembly Tasks in a Robotic Embodiment,†IEEE Trans. Human-Machine Syst., vol. 48, no. 2, pp. 197–206.

DOI: https://doi.org/10.1109/THMS.2017.2782490

Sreeram, S., K. K. Nisha, and R. Jayakrishnan, “Virtual Design Review and Planning Using Augmented Reality and Drones,†in 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS), 2018, pp. 915–918.

DOI: https://doi.org/10.1109/ICCONS.2018.8662919

Zaldívar-Colado, U., S. Garbaya, P. Tamayo-Serrano, X. Zaldívar-Colado, and P. Blazevic, “A mixed reality for virtual assembly,†in 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2017, pp. 739–744.

DOI: https://doi.org/10.1109/ROMAN.2017.8172385

Fernández del Amo, I., J. A. Erkoyuncu, R. Roy, R. Palmarini, and D. Onoufriou, Dec. 2018, “A systematic review of Augmented Reality content-related techniques for knowledge transfer in maintenance applications,†Comput. Ind., vol. 103, pp. 47–71.

DOI: https://doi.org/10.1016/j.compind.2018.08.007

Coleta, G. F. Dela, A. Cardoso, E. A. L. Júnior, and G. F. M. de Lima, “Telecommunications Field Operations Supported by Augmented Reality – a Systematic Review,†in 2019 21st Symposium on Virtual and Augmented Reality (SVR), 2019, pp. 77–83.

DOI: https://doi.org/10.1109/SVR.2019.00028

Fraga-Lamas, P., T. M. FernáNdez-CaraméS, Ó. Blanco-Novoa, and M. A. Vilar-Montesinos, 2018, “A Review on Industrial Augmented Reality Systems for the Industry 4.0 Shipyard,†IEEE Access, vol. 6, pp. 13358–13375.

DOI: https://doi.org/10.1109/ACCESS.2018.2808326

Andersson, N. et al., 2016, “AR-Enhanced Human-Robot-Interaction - Methodologies, Algorithms, Tools,†Procedia CIRP, vol. 44, pp. 193–198.

DOI: https://doi.org/10.1016/j.procir.2016.03.022

Booth, A., D. Papaioannou, and A. Sutton, Systematic Approaches to a Successful Literature Review. 2016, SAGE Publications Ltd.

Aschenbrenner, D. et al., “Comparing Different Augmented Reality Support Applications for Cooperative Repair of an Industrial Robot,†in 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), 2018, pp. 69–74.

DOI: https://doi.org/10.1109/ISMAR-Adjunct.2018.00036

Avalle, G., F. De Pace, C. Fornaro, F. Manuri, and A. Sanna, 2019, “An Augmented Reality System to Support Fault Visualization in Industrial Robotic Tasks,†IEEE Access, vol. 7, pp. 132343–132359.

DOI: https://doi.org/10.1109/ACCESS.2019.2940887

Guhl, J., S. Tung, and J. Kruger, “Concept and architecture for programming industrial robots using augmented reality with mobile devices like microsoft HoloLens,†in 2017 22nd IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), 2017, pp. 1–4.

DOI: https://doi.org/10.1109/ETFA.2017.8247749

Zou, W., M. Andulkar, and U. Berger, “Development of Robot Programming System through the use of Augmented Reality for Assembly Tasks,†in ISR 2018; 50th International Symposium on Robotics, 2018, pp. 1–7.

Linnerud, Å. S., R. Sandøy, and L. E. Wetterwald, “CAD-based system for programming of robotic assembly processes with human-in-the-loop,†in 2019 IEEE 28th International Symposium on Industrial Electronics (ISIE), 2019, pp. 2303–2308.

DOI: https://doi.org/10.1109/ISIE.2019.8781385

Costa, C. M. et al., “Modeling of video projectors in OpenGL for implementing a spatial augmented reality teaching system for assembly operations,†in 2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), 2019, pp. 1–8.

DOI: https://doi.org/10.1109/ICARSC.2019.8733617

Blankemeyer, S., R. Wiemann, L. Posniak, C. Pregizer, and A. Raatz, 2018, “Intuitive Robot Programming Using Augmented Reality,†Procedia CIRP, vol. 76, pp. 155–160.

DOI: https://doi.org/10.1016/j.procir.2018.02.028

Mourtzis, D., V. Zogopoulos, and E. Vlachou, 2017, “Augmented Reality Application to Support Remote Maintenance as a Service in the Robotics Industry,†Procedia CIRP, vol. 63, pp. 46–51.

DOI: https://doi.org/10.1016/j.procir.2017.03.154

De Pace, F., F. Manuri, A. Sanna, and D. Zappia, Mar. 2019, “A Comparison Between Two Different Approaches for a Collaborative Mixed-Virtual Environment in Industrial Maintenance,†Front. Robot. AI, vol. 6.

DOI: https://doi.org/10.3389/frobt.2019.00018

Alves, J., B. Marques, M. Oliveira, T. Araújo, P. Dias, and B. S. Santos, “Comparing Spatial and Mobile Augmented Reality for Guiding Assembling Procedures with Task Validation,†in 2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), 2019, pp. 1–6.

DOI: https://doi.org/10.1109/ICARSC.2019.8733642

Downloads

Published

2020-12-07

Issue

Vol. 83 No. 1: January 2021

Section

Science and Engineering

License

Copyright of articles that appear in Jurnal Teknologi belongs exclusively to Penerbit Universiti Teknologi Malaysia (Penerbit UTM Press). This copyright covers the rights to reproduce the article, including reprints, electronic reproductions, or any other reproductions of similar nature.

How to Cite

A REVIEW ON AUGMENTED REALITY TRACKING METHODS FOR MAINTENANCE OF ROBOTS. (2020). Jurnal Teknologi (Sciences & Engineering), 83(1), 37-43. https://doi.org/10.11113/jurnalteknologi.v83.14907