FRAMEWORK OF CONTROLLING 3D VIRTUAL HUMAN EMOTIONAL WALKING USING BCI
DOI:
https://doi.org/10.11113/jt.v75.5062Keywords:
Component, emotion recognition, EEG, BCI, frameworkAbstract
A Brain-Computer Interface (BCI) is the device that can read and acquire the brain activities. A human body is controlled by Brain-Signals, which considered as a main controller. Furthermore, the human emotions and thoughts will be translated by brain through brain signals and expressed as human mood. This controlling process mainly performed through brain signals, the brain signals is a key component in electroencephalogram (EEG). Based on signal processing the features representing human mood (behavior) could be extracted with emotion as a major feature. This paper proposes a new framework in order to recognize the human inner emotions that have been conducted on the basis of EEG signals using a BCI device controller. This framework go through five steps starting by classifying the brain signal after reading it in order to obtain the emotion, then map the emotion, synchronize the animation of the 3D virtual human, test and evaluate the work. Based on our best knowledge there is no framework for controlling the 3D virtual human. As a result for implementing our framework will enhance the game field of enhancing and controlling the 3D virtual humans’ emotion walking in order to enhance and bring more realistic as well. Commercial games and Augmented Reality systems are possible beneficiaries of this technique.
References
Zhuang, Y., Y. Pan and J. Xiao. 2008. A Modern Approach to Intelligent Animation. China: Springer.
Rauterberg, M. and M. Combetto. 2006. Entertainment Computing-ICEC. In Proceedings of the 5th International Conference Entertainment Computing (ICEC 2006). Cambridge, UK. 20-22 Sept. 2006. 420.
Basori, A. H., D. Daman, A. Bade, M. S. Sunar and M. Saari. 2008. The Feasibility of Human Haptic Emotion as a Feature to Enhance Interactivity and Immersiveness on Virtual Reality Game. In Proceedings of the 7th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry. Singapore. 8-9 Dec. 2008. 37.
Basori, A. H., D. Daman, A. Bade and M. S. Sunar. 2008. The Potential of Human Haptic Emotion as a Technique for Virtual Human Characters Movement to Augment Interactivity in Virtual Reality Game. The International Journal of Virtual Reality. 7(2): 27-32.
Marks, S., J. Windsor and B. Wünsche. 2007. Evaluation of Game Engines for Simulated Surgical Training. In Proceedings of the 5th International Conference on Computer Graphics and Interactive Techniques in Australia and Southeast Asia. Perth, Western Australia. 1-4 Dec. 2007. 273-280.
Adamo-Villani, N. 2007. A Virtual Learning Environment for Deaf Children: Design and Evaluation. International Journal of Human and Social Sciences. 2(2): 123-128.
Greitzer, F.L., O.A. Kuchar and K. Huston. 2007. Cognitive Science Implications for Enhancing Training Effectiveness in a Serious Gaming Context. Journal on Educational Resources in Computing (JERIC). 7(3): 2.
Greitzer, F.L., O.A. Kuchar and K. Huston. 2007. Cognitive Science Implications for Enhancing Training Effectiveness in a Serious Gaming Context. Journal on Educational Resources in Computing (JERIC). 7(3): 2.
Abásolo, M. J. and J. M. Della. 2007. Magallanes: 3d Navigation for Everybody. In Proceedings of the 5th International Conference on Computer Graphics and Interactive Techniques in Australia and Southeast Asia. Perth, Westren Australia. 1-4 Dec. 2007. 135-142.
Takamura, Y., N. Abe, K. Tanaka, H. Taki and S. He. 2006. A Virtual Billiard Game with Visual, Auditory and Haptic Sensation. In Pan, Z., R. Aylett, H. Diener, X. Jin, S. Göbel, L. Li, (eds.). Technologies for E-Learning and Digital Entertainment. 700-705.
Magnenat, T.N. and U. Bonanni. 2006. Haptics in Virtual Reality and Multimedia. Multimedia, IEEE. 13(3): 6-11.
Bailenson, J. N., N. Yee, S. Brave, D. Merget and D. Koslow. 2007. Virtual Interpersonal Touch: Expressing and Recognizing Emotions through Haptic Devices. Human–Computer Interaction. 22(3): 325-353.
Padrew, L. 2008. Character Emotion in 2D and 3D Animation. Boston, MA: Thomson/Course Technology.
Yin, K., K. Loken and M. van de Panne. 2007. SIMBICON: Simple Biped Locomotion Control. In ACM Transactions on Graphics (TOG). 26(3): 105.
Sok, K. W., M. Kim and J. Lee. 2007. Simulating Biped Behaviors from Human Motion Data. ACM Transactions on Graphics (TOG). 26(3): 107.
Oshita, M. 2011. Multi-Touch Interface for Character Motion Control. ACM/Eurographics Symposium on Computer Animation 2011. Vancouver, Canada. 5-6 Aug. 2011.
Tanaka, H., M. N. Zamri and M. Oshita. 2012. Interactive Human Style Deformation for Cyclic Motions. In Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry. Singapore. 2-4 Dec. 2012. 348.
Oshita, M., R. Yamanaka, M. Iwatsuki, Y. Nakatsuka and T. Seki. 2012. Easy-to-Use Authoring System for Noh (Japanese Traditional) Dance Animation. In ACM SIGGRAPH 2012 Posters. Darmstadt. 25-27 Sept. 2012. 45-52.
Acosta, E. J. 2001. Haptic Virtual Environment. Master Thesis. Texas Tech. University.
Basori, A. H., A. Bade, M. S. Sunar, D. Daman and N. Saari. 2011. Natural Interaction for Realistic Emotional Facial Expression of Avatar using Brain Computer Interface and Hand Gestures. Journal of.Computer and Information Technology. 1(2011): 3.
Bogdanovych, A. 2007. Virtual Institutions. Phd. Thesis. University of Technology Sydney.
Yahaya, R. 2007. Immersive Virtual Reality Learning Environment: Learning Decision-Making Skills in A Virtual Reality-Enhanced Learning Environment. PhD Thesis. Queensland University of Technology.
Bork, A. 1991. Learning in Twenty-First Century:Interactive Multimedia Technology. Interactive Multimedia Learning Environments. NATO ASI Series. 93: 2-18.
Hutchison, A. 2007. Back to the Holodeck: New Life for Virtual Reality? In Proceedings of the 2nd International Conference on Digital Interactive Media in Entertainment and Arts. Perth, Western Australia. 19-21 Sept. 2007. 98-104.
Sourina, O., Y. Liu and M.K. Nguyen. 2012. Real-Time EEG-based Emotion Recognition for Music Therapy. Journal on Multimodal User Interfaces. 5(1-2): 27-35.
Zagalo, N. and A. Torres. 2008. Character Emotion Experience in Virtual Environments. The Visual Computer. 24(11): 981-986.
Traum, D. 2008. Talking to Virtual Humans: Dialogue Models and Methodologies for Embodied Conversational Agents. In Modeling Communication with Robots and Virtual Humans. Lecture Notes in Computer Science. 4930: 296-309.
Rauterberg, M. and M. Combetto. 2006. Entertainment Computing-ICEC 2006. In Proceedings of the 5th International Conference. Cambridge, UK. 20-22 Sept. 2006. 4161.
GarcÃa-Rojas, A., F. Vexo and D. Thalmann. 2007. Semantic Representation of Individualized Reaction Movements for Virtual Human. International Journal of Virtual Reality. 6(1): 25-32.
lamare, F. D. 2005. Game Design Perspective. Graphics Series.
Russell, J. A. 1980. A Circumplex Model of Affect. Journal of Personality and Social Psychology. 39(6): 1161-1178.
Michael, D. R. and S. L. Chen. 2005. Serious Games: Games that Educate, Train, and Inform. Thomson.
Zagalo, N., A. Barker and V. Branco. 2004. Story Reaction Structures to Emotion Detection. In Proceedings of the 1st ACM Workshop on Story Representation, Mechanism and Context. New York, USA. 15 Oct. 2004. 33-38.
Wolpaw, J. R., N. Birbaumer, D. J. McFarland, G. Pfurtscheller and T. M. Vaughan. 2002. Brain-Computer Interfaces for Communication and Control. Clinical Neurophysiology. 113(6): 767-791.
Vaughan, T. M., W. J. Heetderks, L. J. Trejo, W. Z. Rymer, M. Weinrich, M. M. Moore and J. R. Wolpaw. 2003. Brain-Computer Interface Technology: A Review of the Second International Meeting. IEEE Transactions on Neural Systems and Rehabilitation Engineering: A Publication of the IEEE Engineering in Medicine and Biology Society. 11(2): 94.
Petrantonakis, P. C. and L. J. Hadjileontiadis. 2010. Emotion Recognition from Brain Signals using Hybrid Adaptive Filtering and Higher Order Crossings Analysis. IEEE Transactions on Affective Computing. 1(2): 81-97.
Yisi, L., O. Sourina and M. K. Nguyen. 2010. Real-Time EEG-based Human Emotion Recognition and Visualization. In Proceedings of the 2010 International Conference on Cyberworlds (CW '10). Nanyang Technological University Singapore, Singapore. 20-22 Oct. 2010. 262-269.
Davidson, R. J., P. Ekman, C. D. Saron, J. A. Senulis and W.V. Friesen. 1990. Approach-Withdrawal and Cerebral Asymmetry: Emotional Expression and Brain Physiology. Journal of Personality and Social Psychology. 58(2): 330-341.
Kostyunina, M. B. and M. A. Kulikov. 1996. Frequency Characteristics of EEG Spectra in the Emotions. Neuroscience and Behavioral Physiology. 26(4): 340-343.
Higuchi, T. 1988. Approach to an Irregular Time Series on the Basis of the Fractal Theory. Physica D: Nonlinear Phenomena. 31(2): 277-283.
Wang, Q., O. Sourina and M. K. Nguyen. 2011. Fractal Dimension based Neurofeedback in Serious Games. The Visual Computer. 27(4): 299-309.
Orozco, H., F. Ramos, M. Ramos and D. Thalmann. 2011. An Action Selection Process to Simulate the Human Behavior in Virtual Humans with Real Personality. The Visual Computer. 27(4): 275-285.
McCauley, L., S. Franklin and M. Bogner. 2000. An Emotion-based “Conscious†Software Agent Architecture. In Affective Interactions. 1814: 107-120.
Bosse, T., M. Pontier and J. Treur. 2007. A Dynamical System Modelling Approach to Gross Model of Emotion Regulation. In Proceedings of the 8th International Conference on Cognitive Modeling (ICCM). Ann Arbor, Michigan. 27-29 July 2007. 187-192.
Mandryk, R. L., M. S. Atkins and K. M. Inkpen. 2006. A Continuous and Objective Evaluation of Emotional Experience with Interactive Play Environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Montreal, Canada. 22-27 Apr. 2006. 1027-1036.
Fulton, B. and M. Medlock. 2003. Beyond Focus Groups: Getting More Useful Feedback from Consumers. [Online]. From: http://download.microsoft.com/download/e/7/4/e74fdd51-fcf2-4429-991d-e446718a49c1/mgsut_FM03.doc.
Lazzaro, N. 2004. Why We Play Games: Four Keys to More Emotion Without Story. XEO Analysis. 1-8.
Downloads
Published
Issue
Section
License
Copyright of articles that appear in Jurnal Teknologi belongs exclusively to Penerbit Universiti Teknologi Malaysia (Penerbit UTM Press). This copyright covers the rights to reproduce the article, including reprints, electronic reproductions, or any other reproductions of similar nature.
