Self-attention improves EEG workload classification in one multitasking study

Adding a lightweight self-attention layer to a hybrid electroencephalography analysis pipeline raised subject-grouped, cross-validated accuracy from 80.73% to 91.33% for participants performing the SIMKAP multitasking task in the STEW dataset. The study included 45 participants with usable subjective workload ratings. In the active-task condition, mean area under the receiver operating characteristic curve, or AUC, increased from 0.910 for convolutional neural network features alone to 0.937 with attention. The accuracy gain favored the attention model in all five folds, although the benefit was smaller and more variable for resting-state recordings.

The experiment analyzed the publicly available STEW dataset, which contains 14-channel EEG recorded with an Emotiv EPOC headset at 128 Hz. Each of 48 participants completed a 2.5-minute resting condition and a 2.5-minute SIMKAP session involving visual matching alongside auditory arithmetic and data-retrieval prompts, some with delayed responses. Participants rated perceived workload afterward on a one-to-nine scale, with usable ratings available for 45 people. Researchers classified low and high workload separately within each condition using condition-specific median splits, then assigned every two-second EEG window the label associated with the participant’s recording.

ACXNet++ builds on the earlier ACXNet design by retaining its autoencoder, convolutional neural network and XGBoost classifier while introducing a four-head self-attention module between the CNN and the final classifier. A fully connected autoencoder first compressed each flattened window, containing 14 channels and 256 samples, from 3,584 dimensions to a 128-dimensional representation. Its encoder used fully connected layers with ReLU activations, while the decoder reconstructed the normalized input through a sigmoid output. Two one-dimensional convolutional blocks then converted the latent vector into a sequence of 32 feature tokens, each containing 64 values. Instead of pooling every token with equal weight, the attention layer calculated data-dependent relationships among them for each input window, allowing the model to emphasize segments that appeared more relevant to the workload decision.

Attention-generated values were pooled into a 32-dimensional representation and passed either alone or together with 36 handcrafted EEG features to XGBoost. The handcrafted block combined means and standard deviations across the 14 channels for temporal, spectral and nonlinear measures, including signal amplitude, zero-crossing rate, skewness, kurtosis, spectral entropy, power in the delta, theta, alpha, beta and low-gamma bands, approximate entropy, the Hurst exponent, an autoregressive coefficient and a wavelet-detail proxy. The layer also used a residual connection and layer normalization, although the study did not establish that these components independently stabilized the attention output. A sigmoid classification head was attached only while pretraining the feature extractor; all reported classification results came from the downstream XGBoost model rather than that auxiliary head.

A major methodological finding emerged during the study’s initial evaluation, which used a single subject-independent 70/15/15 train, validation and test split of 31, seven and seven participants. Several configurations reached 100% training accuracy. Yet the handcrafted-feature classifier obtained a test AUC of just 0.033 for the resting condition, meaning its probabilities were not merely uninformative but confidently anti-correlated with the held-out labels. The researchers identified subject-identity memorization as the likely mechanism: all windows from one participant carried the same condition-level label, and overlapping windows were highly autocorrelated, allowing a sufficiently flexible model to recognize participant-specific patterns rather than learn transferable workload signals. Because the split itself kept participants separate, the problem was not conventional subject leakage across training and test sets; it was memorization of identities and their associated labels within a small held-out-subject evaluation.

To address the failure, the researchers replaced the single split with subject-level stratified five-fold group cross-validation. All windows from a participant remained within one fold, ensuring that a model was always tested on participants it had not encountered during training. Within each training fold, another 15% of participants was reserved for early stopping. The final XGBoost was also regularized more strongly, with a maximum tree depth of four, row and column subsampling ratios of 0.7, a minimum child weight of five, L1 regularization of 0.5, L2 regularization of 2.0 and early stopping after 15 rounds without validation improvement. Performance was summarized as means and standard deviations across the five folds, while aggregate ROC curves and confusion matrices were calculated from out-of-fold predictions.

Under the revised protocol, attention produced its clearest gain during active multitasking. CNN features alone averaged 80.73% accuracy and an AUC of 0.910 on SIMKAP, compared with 91.33% accuracy and an AUC of 0.937 for CNN features with attention. The attention version performed better in every fold. A paired Wilcoxon signed-rank test returned a two-sided p-value of 0.0625, the smallest value mathematically possible with five paired observations and therefore an indicator of unanimous directional agreement rather than a marginal effect. The fully fused model, which combined attention-enhanced CNN features with handcrafted descriptors, did not exceed the unfused attention model on SIMKAP: its mean accuracy was 87.59%, even though its training accuracy rose from 98.32% to 99.34%. The authors interpreted the reduction as consistent with additional handcrafted variables giving the classifier room to fit subject-specific idiosyncrasies when the learned representation was already strong.

The resting condition told a different story. CNN features alone achieved 69.05% mean accuracy and an AUC of 0.803, while CNN-plus-attention features averaged 69.92% accuracy and an AUC of 0.848, with substantially greater variation across folds. The clearest resting-state improvement instead came from concatenating attention features with handcrafted measures: the full ACXNet++ configuration reached 75.59% accuracy, 73.18% F1 score and an AUC of 0.827. Handcrafted features alone averaged 45.89% accuracy for the resting condition, while raw autoencoder latents averaged 50.75%. Across the five folds, the full model consistently outperformed these simpler baselines, again reaching the Wilcoxon test’s minimum attainable p-value in each comparison. The results therefore support learned CNN representations for this dataset, but they do not establish that attention improves every workload classification condition.

Interpretability analyses indicated that the learned representation carried most of the discriminative signal. In representative fitted models, CNN-plus-attention features dominated XGBoost’s top feature-importance rankings in both conditions, while measures such as mean spectral power appeared lower down. Visualization of the 32-by-32 attention matrix for one low-workload and one high-workload SIMKAP window showed that attention was concentrated on selected token pairs rather than distributed uniformly, indicating that the layer was not simply reproducing ordinary pooling. This was a qualitative inspection of two examples, however, rather than a formal analysis across the participant population. A separate model-free comparison of theta and beta power in 15 participants also found channel-dependent differences between resting and active conditions, providing limited independent support for the presence of distinct spectral content in the two conditions.

The authors cautioned that the findings are confined to one dataset, one 14-channel headset, 45 participants with usable ratings and two task conditions. Leave-one-subject-out evaluation, external datasets and a broad sensitivity analysis of attention-head count, latent dimensions and alternative attention placements were not performed. Manual independent-component analysis for eye-blink and muscle artifacts was also omitted because the original component-rejection decisions could not be reproduced, leaving residual artifacts in the recordings. The models nevertheless retained a sizable gap between training and test performance, including roughly 99% training accuracy and 88% mean test accuracy for the full model on SIMKAP, indicating that some subject-specific structure remained. Direct comparison with the original ACXNet results is therefore limited: the earlier work reported 92.10% accuracy for SIMKAP and 89.94% for No-Task from one split, whereas the cross-validated full model here produced lower means of 87.59% and 75.59%. The protocols differed in labeling, feature construction, artifact handling and evaluation design, so the values are not like-for-like.

Subject of Research: Evaluation of attention-enhanced deep learning for EEG mental workload classification in the STEW dataset

Article Title: ACXNet++ an attention augmented hybrid deep learning framework for cross task mental workload estimation from EEG neural manifolds

Article References: Sundaravadivel, P., & Abinaya, G. (2026). ACXNet++ an attention augmented hybrid deep learning framework for cross task mental workload estimation from EEG neural manifolds. Discover Artificial Intelligence, 6(1), Article 1091. https://doi.org/10.1007/s44163-026-02154-7

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02154-7

Keywords: EEG, mental workload, self-attention, deep learning, XGBoost, cross-subject validation, SIMKAP, STEW dataset, ACXNet++, subject memorization, ACXNet, attention