DYNAFEAT: NOVEL FRAMEWORK FOR DYNAMIC FUSION FOR EMOTION ADAPTATION AND TRANSITION IN REAL-TIME USING PHYSIOLOGICAL DATA

Author

Namazbai Ishmakhametov, Kennesaw State UniversityFollow

Semester of Graduation

Fall 2025

Degree Type

Dissertation/Thesis

Degree Name

Data Science and Analytics

Department

School of Data Science and Analytics

Committee Chair/First Advisor

Dr. Sylvia Bhattacharya

Second Advisor

Dr. Sherry Ni

Third Advisor

Dr. Herman Ray

Fourth Advisor

Dr. Jason Metcalfe

Abstract

Current affective computing approaches largely treat emotional states as static events, such as "happy" or "sad," ignoring the continuous nature of real-world emotional transitions. This static paradigm fails to capture how emotion emerges, evolves, and dissipates over time. This dissertation proposes DynaFEAT (Dynamic Fusion for Emotion Adaptation and Transition), a comprehensive methodological framework utilizing a dynamic fusion of multimodal signals to analyze and decode these emotional changes in a continuous setting.

First, we present DEJA-VU (Dynamic Emotional Journey Analysis in Virtual Universe), a validated multimodal dataset (EEG, ECG, EMG, GSR) collected from 28 participants in Virtual Reality, specifically engineered to induce systematic transitions between emotional states. Second, utilizing this data, we address the century-old James-Lange vs. Cannon-Bard debate on physiological precedence. By applying Granger Causality and the Directionality Asymmetry Index (DAI), we demonstrate that precedence is not a fixed hierarchy but a context-dependent dynamic (Double Dissociation): affiliative states (Joy) exhibit a "Body-First" reflexive loop, while threat states (Fear) utilize a "Brain-First" command structure. Finally, we decode the "grammar of emotional transitions" using Dynamic Time Warping and Deep Learning (Bi-GRU). We identify distinct topological signatures for specific transition types and propose the Decomposition Hypothesis, demonstrating that universal emotional signals (U) can be recovered from idiosyncratic physiological baselines (P) through cross-session calibration. This work establishes a new standard for modeling "emotions in motion," shifting the focus of affective systems from static recognition to dynamic regulation.

Keywords: Affective Computing, Virtual Reality, Multimodal Fusion, Deep Learning, Psychophysiology

Comments

I was funded by DEVCOM ARL grant W911NF2020205