A preliminary study evaluating a video game framework has shown that dynamically adapting in-game weather based on player facial affect can increase immersion while reducing negative affect. In the trial, participants experienced a 40.67 percent increase in immersion alongside a 32.05 percent reduction in negative affect. Published in the journal Multimedia Tools and Applications, the research by Chetan Mudlapur and Om Prakash Singh introduces an artificial intelligence architecture named A.U.R.A., short for Affective User Response Atmosphere. The project sets out to invert the long-standing design paradigm of interactive media, in which game designers traditionally employ weather as a predetermined atmospheric and narrative device crafted to evoke specific emotional reactions in players. Under the authors’ proposed architecture, player emotion itself becomes the active input driving virtual weather conditions dynamically.
To realize this inverted interaction loop, the researchers developed an affect-aware processing pipeline centered on facial emotion recognition. At the core of the A.U.R.A. framework is a custom convolutional neural network architecture that was trained on a filtered version of the FER-2013 facial expression dataset. This networking configuration transmits real-time emotion recognition inferences directly into the Godot game engine, specifically version 4.4.1. By relaying facial affect data via high-speed WebSockets, the framework continuously translates the user’s detected emotional state into responsive modifications of the simulated environmental conditions.
For their experimental evaluation, the team engineered a dedicated two-dimensional demonstration video game in Godot. Within this custom demo game, a fuzzy logic controller was incorporated to dynamically scale enemy difficulty in tandem with the player’s session. Technical benchmarking confirmed that the entire pipeline delivers high performance during active play, maintaining a stable frame rate of 119 to 120 frames per second in the two-dimensional test environment. In addition to these two-dimensional benchmark results, the investigators demonstrated that the framework possesses three-dimensional scalability, indicating its technical capability to extend into three-dimensional multimedia virtual environments.
The user-experience outcomes were gathered through a preliminary evaluation conducted with a sample of 32 participants (n=32). While the observed 40.67 percent boost in player immersion and the 32.05 percent decrease in negative affect confirm the basic viability of using real-time facial affect pipelines to build personalized virtual worlds, the authors note the preliminary nature of the trial.
Subject of Research: Affect-driven dynamic video game weather systems using convolutional neural networks
Article Title: Facial affect-driven video game weather system using custom CNN
Article References: Mudlapur, C., & Singh, O. P. (2026). Facial affect-driven video game weather system using custom CNN. Multimedia Tools and Applications, 85(11), Article 832. https://doi.org/10.1007/s11042-026-22005-y
Image Credits: AI Generated
DOI: 10.1007/s11042-026-22005-y
Keywords: affective computing, video games, facial emotion recognition, convolutional neural networks, A.U.R.A, Godot engine, dynamic difficulty adjustment, fuzzy logic, Facial, affect-driven, video, game
News Source: Denise Maddox. (November 9, 2026). AI System Dynamically Adapts Video Game Weather to Player Facial Affect. Scienmag.



