Affective Virtual Reality Gaming for Autism
Abstract
Emotional impairment is one of the common symptoms of many mental diseases. Being able to learn the emotional reactions from subjects using nonintrusive human-computer interactions (HCI) would provide a novel and efficient approach to assist existing intervention and therapy. Psychologists conducted research using virtual reality (VR) as a tool for exposure starting from decades ago. However, early VR equipment was cumbersome in size and inefficient, which can only be used to simulate limited scenes, such as car driving and phobia scenarios. With the evolution of affordable and portable VR hardware, we are now able to design systematic VR games that can precisely control variables for different stimuli and testing cases. Furthermore, the VR nowadays can serve not only as exposure methods, but also real games that seamlessly and nonintrusively interact with users. The experience of immersion and presence has made VR naturally suitable for triggering strong emotions. Moreover, the study of affective computing, known as the study and development of systems and devices that can recognize, interpret, process, and simulate human affects, is a trendy and challenging topic in HCI field. Given current research and potential development, affective computing is seeking to develop emotional intelligence in machines. Integrating the concept of affective computing into game design and development will lead to a new type of serious games that interact with users’ emotions. This chapter reviews the methodologies commonly used in affective computing and related research projects using VR exposure as an intervention for people with special needs. It also describes a series of studies conducted to collect and analyze data. Our goal is to propose a game framework that recognizes users’ emotional reactions in a multimodal approach, which, with ideal expectation, adapts according to the fluctuation of the users’ emotional states dynamically. The system provides a guideline for affective gaming design for mental healthcare purposes. The application of the framework is to assist in intervention for autistic spectrum disorder and can be extended to other emotion-related mental illness.