AI Reads Brain and Body Signals to Predict How People Rate Car Cockpits

The work, published in Frontiers in Human Neuroscience, tackles a long-standing problem in Kansei engineering, the discipline that translates subjective product impressions into design-relevant information.

Traditionally, Kansei evaluation depends on self-reported judgments, which are vulnerable to contextual influences, recall bias, and the timing of assessment. Physiological measurements offer a complementary window into affective responses, capturing attention allocation, arousal, and task engagement as products are actually viewed. The difficulty has always been integration: EEG, eye tracking, and peripheral signals differ in units, dimensionality, noise characteristics, and temporal resolution, and combining them with only a handful of participants risks unstable, inflated performance estimates.

The new study, led by Ming Deng of Northeastern University in Shenyang, addressed both challenges at once. Fifty healthy adults — 25 men and 25 women, aged 20 to 30 — viewed three high-resolution renderings of automobile cockpit designs that differed in proportion, color treatment, and interface complexity. Each trial included a 500-millisecond fixation period, a six-second viewing interval, and self-paced ratings on five-point semantic-differential scales anchored by unattractive–attractive, conventional–innovative, and unfriendly–friendly. The experiment yielded 150 participant–stimulus trials, all of which passed prespecified quality-control criteria and were retained for analysis.

EEG was recorded with a 64-electrode Brain Products actiCHamp system, gaze with a Tobii Pro Spectrum eye tracker, and peripheral signals with a Biopac MP160, with Lab Streaming Layer used to synchronize acquisition across devices. Notably, the researchers excluded conventional frequency-domain heart-rate variability measures such as LF power, HF power, and the LF/HF ratio, because a single cycle at the lower LF boundary lasts 25 seconds, far exceeding the six-second viewing window. Only two ultra-short-window ECG descriptors, RMSSD and SDNN, survived into the final predictor set.

The distinctive ingredient in GAMAF is generative augmentation. Because psychophysiological experiments typically involve few independent participants, the team used a conditional Wasserstein generative adversarial network with gradient penalty to synthesize additional training samples in feature space, conditioned on the observed ratings. The augmentation ratio proved to be a genuine trade-off rather than a free lunch: increasing synthetic data from 50 to 200 percent steadily improved both distributional similarity and downstream utility, with maximum mean discrepancy falling from 0.045 to 0.012 and auxiliary F1 rising from 0.695 to 0.808. But pushing augmentation to 300 and 400 percent degraded both measures, indicating that the 200 percent ratio is an empirical choice for this dataset, not a universal rule.

On top of the augmented training distribution, GAMAF layers three architectural components: cross-modal attention that lets each modality’s representation query the other two, a graph attention layer treating the three modality representations as nodes in a computational graph, and an adaptive gating network that assigns target-dependent weights to the fused representation. Validation was deliberately strict — five outer cross-validation folds grouped by participant, with all preprocessing, feature selection, generator fitting, and hyperparameter tuning confined to the training partitions so that no held-out participant’s data could leak into model construction.

The headline numbers are striking. GAMAF achieved mean outer-fold R² values of 0.810 for attractiveness, 0.850 for innovativeness, and 0.840 for friendliness, each with a standard deviation of approximately 0.016. These exceeded the Multimodal Transformer baseline’s 0.75, 0.78, and 0.76, and across the three targets GAMAF reduced aggregate mean absolute error and root mean squared error by roughly 29.6 and 29.7 percent. Ablation analyses supported the integrated design: removing generative augmentation dropped mean R² from 0.833 to 0.700, removing graph interaction yielded 0.733, and replacing adaptive gating with equal weights produced the largest reduction, to 0.637.

Perhaps most intriguing are the target-dependent patterns in the learned modality weights. Eye tracking received the largest weight for attractiveness judgments (63.1 percent), EEG dominated for innovativeness (72.5 percent), and peripheral physiology carried the most weight for friendliness (71.3 percent). The authors are careful about interpretation: because graph propagation mixes cross-modal information before gating, these percentages describe the model’s weighting of modality-labeled latent representations, not exclusive sensor-level contributions or physiological causation. Descriptive ANOVA summaries reinforced the picture, with the largest category separation observed for skin conductance response amplitude, followed by frontal alpha asymmetry and pupil diameter.

The study is equally candid about its limits. Performance comparisons remain descriptive because baseline fold variability and participant-level uncertainty were not quantified, and the five outer folds share training observations, so they do not constitute five independent replications. The participant-wise validation estimates prediction for new individuals viewing this same three-stimulus set; it does not establish transfer to unseen cockpit designs. The five-point ratings are ordinal but were treated as approximately interval-scaled, and the physiological statistics are descriptive summaries rather than confirmatory population-level inference, since repeated observations within participants violate independence assumptions.

What the framework demonstrably does is establish correspondence between concurrent physiological signals and people’s expressed product impressions — not replace subjectivity with an objective ground truth. The authors envision predicted Kansei ratings supporting early-stage screening of design concepts, potentially feeding a future human-in-the-loop workflow in which a generative model proposes cockpit candidates, users view them during multimodal recording, and designers use the estimated impressions to guide the next iteration. That closed-loop vision remains prospective: the present study evaluates the decoding stage only, and broader participant samples, unseen designs, matched uncertainty estimates, and behavioral validation are needed before such applications become reality.

Subject of Research: Multimodal psychophysiological prediction of subjective product-impression ratings in automobile cockpit evaluation

Article Title: Generative-augmented multimodal neuroaffective decoding for product evaluation: toward neurofeedback-driven human-in-the-loop co-design

Article References: Deng, M. (2026). Generative-augmented multimodal neuroaffective decoding for product evaluation: toward neurofeedback-driven human-in-the-loop co-design. Frontiers in Human Neuroscience, 20, Article 1997654. https://doi.org/10.3389/fnhum.2026.1997654

Image Credits: AI Generated

DOI: 10.3389/fnhum.2026.1997654

Keywords: Kansei engineering, GAMAF, EEG, eye tracking, multimodal fusion, generative augmentation, cWGAN-GP, affective computing, product design, cross-validation, automotive cockpit, neuroaffective computing