A REVIEW OF 3D GESTURE INTERACTION FOR HANDHELD AUGMENTED REALITY
DOI:
https://doi.org/10.11113/jt.v78.6923Keywords:
Gesture Interaction, Handheld, Augmented RealityAbstract
Interaction for Handheld Augmented Reality (HAR) is a challenging research topic because of the small screen display and limited input options. Although 2D touch screen input is widely used, 3D gesture interaction is a suggested alternative input method. Recent 3D gesture interaction research mainly focuses on using RGB-Depth cameras to detect the spatial position and pose of fingers, using this data for virtual object manipulations in the AR scene. In this paper we review previous 3D gesture research on handheld interaction metaphors for HAR. We present their novelties as well as limitations, and discuss future research directions of 3D gesture interaction for HAR. Our results indicate that 3D gesture input on HAR is a potential interaction method for assisting a user in many tasks such as in education, urban simulation and 3D games.
References
Azuma, R. T. et al. 1997. A Survey Of Augmented Reality. Presence. 6(4): 355-385.
Bowman, D. A., Kruijff, E., La Viola Jr, J. J. and Poupyrev, I. 2004. 3D User Interfaces: Theory And Practice. Addison-Wesley.
Zhou, F., Duh, H. B. L., and Billinghurst, M. 2008. Trends in Augmented Reality Tracking, Interaction And Display: A Review Of Ten Years Of Ismar. Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, ser. ISMAR ’08. Washington, DC, USA: IEEE Computer Society, 2008. 193-202.
Hurst W. and van Wezel, C. 2011. Multimodal Interaction Concepts For Mobile Augmented Reality Applications. In Proceedings of the 17th International Conference on Advances in Multimedia Modeling - Volume Part II, ser. MMM’11. Berlin, Heidelberg: Springer-Verlag. 157-167.
Hinckley, K., Pausch, R., Proffitt, D., Patten, J. and Kassell, N. 1997. Cooperative Bimanual Action. Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, ser. CHI ’97. New York, NY, USA: ACM. 27-34.
Rekimoto, J. 1996. Transvision: A Hand-Held Augmented Reality System For Collaborative Design. Proc. Virtual Systems and Multimedia. 85-90.
Mogilev, D., Kiyokawa, K., Billinghurst, M. and Pair, J. 2002. Ar pad: An Interface For Face-To-Face Ar Collaboration. In CHI ’02 Extended Abstracts on Human Factors in Computing Systems, ser. CHI EA ’02. New York, NY, USA: ACM. 654-655.
Billinghurst, M., Kato, H. and Poupyrev, I. 2001. The Magicbook: A Transitional Ar Interface. Computers and Graphics. 25: 745-753.
Moehring, M., Lessig, C. and Bimber, O. 2004. Optical Tracking And Video See-Through Ar On Consumer Cell Phones. Workshop On Virtual And Augmented Reality Of The GI-Fachgruppe AR/VR. Citeseer. 193-204.
Wagner, D. and Schmalstieg, D. 2003. Artoolkit On The Pocketpc Platform. IEEE International Augmented Reality Toolkit Workshop. Darmstadt, Gernany. 14-15.
Kato, I. P. H., Billinghurst, M. and Poupyrev, I. 2000. Artoolkit User Manual, Version 2.33. Human Interface Technology Lab, University of Washington. vol. 2.
Wagner, D., Pintaric, T. and Schmalstieg, D. 2004. The Invisible Train: A Collaborative Handheld Augmented Reality Demonstrator. ACM SIGGRAPH 2004 Emerging Technologies. ACM. 12.
Henrysson, A., Marshall, J. and Billinghurst, M. 2007. Experiments In 3d Interaction For Mobile Phone AR. Proceedings of the 5th International Conference on Computer Graphics and Interactive Techniques in Australia and Southeast Asia, ser. GRAPHITE ’07. New York, NY, USA: ACM. 187-194.
Hurst, W. and van Wezel, C. 2013. Gesture-based Interaction Via Finger Tracking for Mobile Augmented Reality. Multimedia Tools and Applications. 62(1): 233-258.
Baldauf, M., Zambanini, S., Fro ̈hlich, P. and Reichl, P. 2011. Markerless Visual Fingertip Detection For Natural Mobile Device Interaction. Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services, ser. MobileHCI ’11. New York, NY, USA: ACM. 539-544.
Seo, B.-K., Choi, J., Han, J.-H., Park, H. and Park, J.-I. 2008. One-handed Interaction With Augmented Virtual Objects On Mobile Devices. Proceedings of The 7th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry, ser. VRCAI ’08. New York, NY, USA: ACM. 8: 1-6.
Huidong, B., Lei, G. and Billinghurst, M. 2013. Poster: Markerless Fingertip- Based 3D Interaction For Handheld Augmented Reality In A Small Workspace. 3D User Interfaces (3DUI). 2013 IEEE Symposium on, 2013. 129-130.
Fritz, D., Mossel, A. and Kaufmann, H. 2014. Evaluating Rgb+D Hand Posture Detection Methods For Mobile 3d Interaction. Proceedings of the 2014 Virtual Reality International Conference, ser. VRIC ’14. New York, NY, USA: ACM. 27: 1-4.
Bai, H., Lee, G., Ramakrishnan, M. and Billinghurst, M. 2014. One-handed Interaction With Augmented Virtual Objects On Mobile Devices. Proceedings of the 7th ACM SIGGRAPH Asia 2014 Symposium on Mobile Graphics and Interactive Applications, ser. SA MGIA ’14. New York, NY, USA: ACM.
Bai, H., Gao, L., El-Sana, J. and Billinghurst, M. 2013. Free-hand Interaction For Handheld Augmented Reality Using An Rgb-Depth Camera. SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, ser. SA ’13. New York, NY, USA: ACM. 22: 1-4.
Wagner, D. and Schmalstieg, D. 2006. Handheld Augmented Reality Displays. Virtual Reality Conference, 2006. IEEE. 321-321.
Mossel, A., Venditti, B. and Kaufmann, H. 2013. 3dtouch And Homer-S: Intuitive Manipulation Techniques For One-Handed Handheld Augmented Reality. Proceedings of the Virtual Reality International Conference: Laval Virtual. ACM. 12.
Vincent, T., Nigay, L. and Kurata, T. 2013. Precise Pointing Techniques For Handheld Augmented Reality. Human-Computer Interaction– INTERACT 2013. Springer, 2013. 122-139.
Bai, H., Gao, L., El-Sana, J. and Billinghurst, M. 2013. Markerless 3d Gesture-Based Interaction For Handheld Augmented Reality Interfaces. SIGGRAPH Asia 2013 Symposium on Mobile Graphics and Interactive Applications, ser. SA ’13. New York, NY, USA: ACM. 22: 1.
Chun, W. H. and Ho ̈llerer, T. 2013. Real-time Hand Interaction For Augmented Reality On Mobile Phones. Proceedings of the 2013 International Conference on Intelligent User Interfaces, ser. IUI ’13. New York, NY, USA: ACM. 307-314.
Bai, H., Lee, G. A. and Billinghurst, M. 2012. Freeze View Touch And Finger Gesture Based Interaction Methods For Handheld Augmented Reality Interfaces. Proceedings of the 27th Conference on Image and Vision Computing New Zealand, ser. IVCNZ ’12. New York, NY, USA: ACM, 2012. 126-131.
Lu, H. and Li, Y. 2012. Gesture Coder: A Tool For Programming Multi-Touch Gestures By Demonstration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2012. 2875-2884.
Telkenaroglu, C. and Capin, T. 2013. Dual-finger 3d Interaction Techniques for Mobile Devices. Personal Ubiquitous Comput. 17(7): 1551-1572.
Lindeman, R., Lee, G., Beattie, L., Gamper, H., Krishnan Pathinarupothi, R. and Akhilesh, A. 2012. GeoBoids: A Mobile AR Application for Exergaming. 11th IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2012)-Arts, Media, and Humanities Pro- ceedings, Atlanta, Georgia, USA, 2012. 93-94.
Sukan, M., Feiner, S., Tversky, B. and Energin, S. 2012. Quick Viewpoint Switching For Manipulating Virtual Objects In Hand-Held Augmented Reality Using Stored Snapshots. Mixed and Augmented Reality (ISMAR), 2012 IEEE International Symposium on. IEEE, 2012. 217-226.
Kratz, S., Rohs, M., Guse, D., Mu ̈ller, J., Bailly, G. and Nischt, M. 2012. Palmspace: Continuous Around-Device Gestures Vs. Multitouch For 3d Rotation Tasks On Mobile Devices. Proceedings of the International Working Conference on Advanced Visual Interfaces, ser. AVI ’12. New York, NY, USA: ACM, 2012. 181-188.
Lee, G. A., Yang, U., Kim, Y., Jo, D., Kim, K.-H., Kim, J. H. and Choi, J. S. 2009. Freeze-set-go Interaction Method For Handheld Mobile Augmented Reality Environments. In Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology, ser. VRST ’09. New York, NY, USA: ACM, 2009. 143-146.
Roudaut, A., Huot, S. and Lecolinet, E. 2008. Taptap and magstick: Improving One-Handed Target Acquisition On Small Touch-Screens. Proceedings of the Working Conference on Advanced Visual Interfaces, ser. AVI ’08. New York, NY, USA: ACM, 2008. 146-153.
Downloads
Published
Issue
Section
License
Copyright of articles that appear in Jurnal Teknologi belongs exclusively to Penerbit Universiti Teknologi Malaysia (Penerbit UTM Press). This copyright covers the rights to reproduce the article, including reprints, electronic reproductions, or any other reproductions of similar nature.
