Surface electromyography recorded from forearm muscles can recognize static American Sign Language gestures without visual cameras, but gestures sharing similar muscular patterns frequently trigger classification errors that short-term temporal context can partially resolve. In a study published in Frontiers in Neuroscience, researchers Yuanxi Wang, Lun Shu, and Yasuharu Koike from the Institute of Science Tokyo investigated the physiological and computational mechanisms behind myoelectric sign language decoding. By tracking 32 channels of electrical activity across the forearm in ten participants performing 24 static letter gestures, the investigators demonstrated that classification confusions are strongly tied to shared muscular activation patterns. Crucially, the team established that integrating recent signal history over a 700-millisecond window raised recognition accuracy to 90.77 percent in participant-specific models and 89.01 percent in shared multi-user models, narrowing the performance gap for the most difficult hand shapes.
Surface electromyography bypasses visual occlusions entirely by detecting the electrical potentials generated during skeletal muscle contractions beneath the skin of the forearm. However, decoding hand gestures exclusively from forearm signals introduces severe biomechanical ambiguities. Several forearm muscles act simultaneously across multiple digits, mechanical linkages between tendons constrain independent movement, and fine positioning of the thumb and fingers is governed largely by intrinsic hand muscles located entirely within the palm and hand structure rather than the forearm.
Ten right-handed healthy male volunteers aged 23 to 30, all new to American Sign Language, were fitted with a specialized 32-channel dry-electrode sleeve worn on the right forearm. Because the experimental sleeve was fabricated in a single physical size, recruitment was restricted to individuals whose forearm dimensions ensured close, uniform contact across all 32 electrode sites. The sleeve arrangement comprised four circumferential rings spanning from distal to proximal forearm locations, with each ring housing eight evenly spaced electrodes sampled at 500 hertz and synchronized via the Lab Streaming Layer protocol.
During data acquisition, participants maintained a seated posture with their arm held stationary to isolate hand posture from gross arm displacement and motion artifacts. Each volunteer completed four testing trials, performing the 24 static letters in a fixed order five times per trial, with each gesture held for three seconds followed by an equivalent three-second rest interval. Across all ten participants, the protocol logged 4,800 individual gesture repetitions, of which 24 repetitions were discarded following video verification due to incorrect finger or thumb positioning, leaving 4,776 valid repetitions for subsequent analysis. The raw electromyographic recordings underwent offline zero-phase filtering using a 50-hertz notch filter to remove electrical grid interference and a 10-hertz high-pass filter to eliminate low-frequency drift and motion artifacts before feature extraction.
The processing pipeline utilized independent component analysis to transform the filtered 32-channel surface mixtures into a demixed 32-component representation without reducing spatial dimensionality. For classification experiments, the demixing matrix was fitted strictly on training repetitions within each fold of a five-fold cross-validation scheme partitioned at the collection-round level. The researchers subsequently extracted short-time Fourier transforms across 250-millisecond sliding windows with 80 percent overlap, corresponding to a 50-millisecond step between successive frames. Applying a logarithmic magnitude transformation yielded a structured feature tensor of 32 independent components across 33 frequency bins and nine time frames. This tensor was fed into a dual-branch neural network encoder combining parallel convolutional pathways for temporal and frequency dimensions with three Transformer encoder layers.
Evaluating a baseline Single-Window Model on the common target set yielded letter-only recognition accuracies of 86.71 percent with a between-participant standard deviation of 3.88 percentage points in the participant-specific configuration, and 85.23 percent with a between-fold standard deviation of 0.87 percentage points in the shared multi-user setup. An in-depth analysis of per-class performance revealed that single-window errors were overwhelmingly concentrated within nine low-recall letters: A, M, N, S, T, E, O, U, and V. While the remaining 15 letters attained an average recall of 91.87 percent, these nine problematic characters averaged only 79.11 percent. Cosine distance evaluations across standardized signal prototypes identified three distinct clusters of high confusion: a closed-fist cluster consisting of A, M, N, S, and T; a curved-finger pair formed by E and O; and an upright two-finger pair formed by U and V.
The distribution of single-window errors closely mirrored these electromyographic similarities, revealing marked non-random misclassifications. Within the closed-fist group, 88 percent of all erroneous letter predictions fell on other members of the same five-letter cluster, drastically exceeding the 17 percent rate expected under uniform error distribution. Similarly, within-group errors accounted for 34 percent of mistakes for the E and O pair and 50 percent for the U and V pair, against a uniform baseline expectation of 4 percent. Spatial activation mapping across the sleeve geometry revealed that the letter pair M and N exhibited minimal mean signed amplitude divergence across forearm channels, averaging just 0.193 standardized units, whereas visually separated control letters such as L and Y demonstrated substantial muscular divergence across channels averaging 0.638 standardized units.
To explore the latent source space, the authors aligned independent component mixing matrices across all ten participants using the Hungarian algorithm, generating 32 component families with a median inter-participant spatial correlation of 0.83. Smaller pairwise distances in independent component space correlated strongly with higher mutual classification confusion across all 276 letter pairings, exhibiting a mean participant-level Spearman rank correlation of negative 0.464. In specific component families, the distal-flexor family F03 showed significantly higher activation for E than O across all ten individuals, while the proximal-flexor family F32 demonstrated higher activation for U than V. Conversely, M and N shared activation within a proximal flexor component without any statistically significant difference surviving multiple-testing correction, illustrating how distinct thumb postures over a closed fist fail to produce distinguishable forearm source signatures.
To overcome these single-window ambiguities, the investigators evaluated a causal Multi-Window Model that combined the current 250-millisecond window with representations from the nine preceding windows, spanning 700 milliseconds of recent muscle history. Processed through a one-layer causal temporal Transformer with eight attention heads, the contextualized architecture increased letter-only accuracy to 90.77 percent in the participant-specific setting and 89.01 percent in the multi-user setting, representing paired gains of 4.06 and 3.78 percentage points, respectively. Improvements occurred across all ten participants in the per-user setting, ranging from 2.42 to 7.29 percentage points. Gains were especially pronounced among difficult gestures: mean recall across the nine low-recall letters climbed by 5.30 percentage points to 83.92 percent, with individual letters such as T and O improving by 8.25 and 8.02 percentage points.
Comparative evaluations against post-hoc decision aggregation highlighted the underlying mechanism of temporal integration. A ten-window majority-voting control operating on independent single-window predictions achieved 89.27 percent per-user accuracy and 88.17 percent multi-user accuracy, recovering the vast majority of the performance increase. While the learned causal Transformer provided an additional edge of 1.50 percentage points over majority voting, randomly shuffling the order of preceding windows while holding the target window fixed reduced accuracy by only 0.26 percentage points in per-user models and 0.41 percentage points in shared models. These findings indicate that repeated observations of sustained muscle activity over time mitigate physiological motor-unit discharge fluctuations and recording noise, with temporal sequence ordering playing a secondary role.
The authors underscored several boundary conditions governing the practical translation of these findings to wearable neuroprosthetics and consumer interfaces. In a leave-one-subject-out evaluation testing generalization to completely unseen users without independent component analysis, recognition dropped sharply to 33.80 percent letter-only accuracy, illustrating significant cross-user variability in muscle recruitment and sleeve placement. Extending analysis intervals from two to three seconds to encompass gesture initiation and relaxation phases reduced Multi-Window accuracy to 88.80 percent per-user, reflecting transient signal variations during movement transitions. Furthermore, host inference benchmarks demonstrated that caching window embeddings kept median per-step latency to 3.41 milliseconds on an RTX 5090 GPU, whereas four-thread CPU execution averaged 45.16 milliseconds with a 95th-percentile latency of 49.23 milliseconds, approaching the 50-millisecond real-time streaming threshold.
The study demonstrates that integrating short-term temporal history provides a dependable computational method for boosting surface electromyography decoding during static sign language gestures. Nevertheless, the persistence of classification errors among gestures that rely primarily on intrinsic hand muscles underscores an inherent biological limit of forearm-only sensing. The researchers concluded that fully resolving ambiguous hand postures in natural communication will require moving beyond isolated static recordings toward continuous gesture streams, alongside incorporating complementary sensing modalities such as optical hand trackers or force myography wristbands that directly capture palm and thumb biomechanics.
Subject of Research: Surface electromyography and deep learning for American Sign Language gesture recognition
Article Title: Analysis and improvement of surface electromyography-based American Sign Language hand gesture recognition
Article References: Wang, Y., Shu, L., & Koike, Y. (2026). Analysis and improvement of surface electromyography-based American Sign Language hand gesture recognition. Frontiers in Neuroscience, 20, Article 1978950. https://doi.org/10.3389/fnins.2026.1978950
Image Credits: AI Generated
DOI: 10.3389/fnins.2026.1978950
Keywords: surface electromyography, American Sign Language, gesture recognition, Transformer, assistive technology, biomechanics, neural decoding, signal processing, Analysis, improvement, surface, electromyography-based
News Source: Ophelia Keating. (November 12, 2026). Forearm Muscle Signals and Temporal Context Boost American Sign Language Recognition. Scienmag.



