A smart home interaction design framework integrating visual saliency modeling, multimodal interaction, and digital twins simulation has demonstrated high operational accuracy and efficiency across multi-scene domestic environments. In physical cross-validation experiments conducted with twenty human participants, users achieved an average comprehensive efficiency index of 0.862, an average interaction error rate of 5.97 percent, and an average user experience score of 8.18.
To evaluate the reliability and predictive precision of the computational models, the research system completed 450 virtual simulations and compared the resulting data directly with the physical user experiments for cross-validation. The quantitative evaluation revealed that the average error in interaction time between the virtual simulation and real-world experiments was 0.078, with the observed error range spanning between 0.061 and 0.089 across the tested sessions. Furthermore, the average difference in the comprehensive efficiency index remained at 0.018, and the error rate difference was approximately positive 0.8 percent, demonstrating that the virtual prototype maintained stable prediction ability in the test scenarios.
The construction of the verifiable virtual prototype was carried out using the Unity software engine, where the researchers developed three separate digital twin environments representing a living room, a bedroom, and a kitchen. This spatial layout was established to investigate smart home interaction methods aiming at improving interaction efficiency, prediction accuracy, and overall user experience within a multi-scene environment.
To generate an effective, computable, and verifiable virtual prototype, the system introduced multiple distinct technical modeling components tailored to ambient domestic conditions.
For the purpose of design space optimization, the researchers deployed a hybrid computational strategy combining a Genetic Algorithm and Particle Swarm Optimization, designated as the GA-PSO algorithm. This optimization procedure systematically searched through 300 groups of parameter combinations to determine the most effective configuration for the smart home interaction system.
The empirical results of the GA-PSO design space search demonstrated substantial quantitative performance improvements across all tested interaction metrics in the smart home system. Specifically, the algorithmic optimization improved the comprehensive efficiency index from 0.86 to 0.91 and raised the task completion rate from 0.927 to 0.953. At the same time, the system error rate was reduced from 5.4 percent down to 4.3 percent, while the average interaction completion time decreased from 13.2 seconds to 11.7 seconds, confirming the practical efficacy of computational search for parameter refinement.
Cross-scene, cross-task, and cross-modal experimental evaluations further demonstrated that multimodal fusion consistently achieved superior performance values compared to single-modal interaction under the tested metrics. In comparative trials, multimodal integration attained better results in overall operational efficiency, latency delay reduction, and subjective user experience scores across the diverse household tasks and multi-scene settings. The authors reported that combining digital twins, visual saliency modeling, and multimodal fusion offers a quantifiable modeling, verification, and optimization path for the future design and deployment of smart home interaction systems.
Subject of Research: Digital twin modeling and multimodal interaction design for smart homes
Article Title: Digital twins based visual saliency smart home interaction design and virtual real validation through multimodal fusion
Article References: Zhang, F. (2026). Digital twins based visual saliency smart home interaction design and virtual real validation through multimodal fusion. Discover Applied Sciences. https://doi.org/10.1007/s42452-026-09598-7
Image Credits: AI Generated
DOI: 10.1007/s42452-026-09598-7
Keywords: digital twins, visual saliency modeling, multimodal fusion, smart home interaction, virtual prototyping, Unity simulation, user experience, comprehensive efficiency index, light field saliency, sound field modeling, gesture trajectory generation, agent-based simulation, genetic algorithm, particle swarm optimization, task completion rate, cross-modal interaction
News Source: Denise Maddox. (November 11, 2026). Digital Twin Smart Home Model Accurately Predicts Real User Multimodal Interactions. Scienmag.



