Multimodal Deep Learning Screens Lower-Limb Alignment Using 3D Foot Scans

A team of researchers has developed and internally validated an explainable multimodal deep learning framework that utilizes weight-bearing plantar three-dimensional foot scans to evaluate screening-level lower-limb alignment. Investigating whether plantar foot morphology captured by weight-bearing three-dimensional scanning can serve as a viable computational screening feature for proximal lower-limb alignment, the study demonstrates that non-invasive scanning combined with multimodal deep learning can identify alignment-related biomechanical risk factors. The investigators developed this computational framework to support scalable alignment screening, offering a potential non-invasive methodology to identify musculoskeletal loading risks.

Proper lower-limb alignment represents a key determinant of musculoskeletal loading and is closely associated with biomechanical risk pathways for debilitating conditions such as knee osteoarthritis and plantar fasciitis. Despite its clinical importance, routine alignment assessment still relies predominantly on radiography or specialist clinical examination, which substantially limits scalability for early screening programs. To overcome these screening bottlenecks, the investigators examined whether weight-bearing plantar foot morphology can act as an effective computational screening feature for alignment across the proximal lower limb.

To develop and evaluate the computational framework, the research team established a cross-sectional dataset consisting of 167 participants, encompassing both children and adults aged 6 to 60 years. All enrolled participants received standardised plantar three-dimensional foot scans alongside clinical labels established for arch type, hindfoot alignment, and frontal-plane knee alignment. The proposed deep learning architecture jointly learns from plantar scan images and 16 geometric morphological features. These input streams are integrated using a feature-wise attention mechanism designed specifically to prioritise alignment-relevant cues across both data modalities.

Performance of the proposed architecture was evaluated on a held-out test set using accuracy, precision, recall, F1 score, and area under the receiver operating characteristic curve. Ablation studies systematically compared alternative image encoders and multimodal fusion variants, including image-only, morphology-only, and fusion without attention. The attention-based multimodal model demonstrated competitive but task-dependent performance across the different targets, achieving accuracies of 89 percent for arch type, 81 percent for hindfoot alignment, and 76 percent for frontal-plane knee alignment. The comparative ablation experiments indicated that an image-only Vision Transformer attained the highest accuracy for arch classification, whereas a Vision Transformer configuration operating without attention achieved slightly higher accuracy and macro-F1 score for knee alignment.

To examine how the networks processed foot scans, the authors assessed model interpretability using class activation mapping. These activation maps consistently highlighted clinically relevant plantar regions associated with specific alignment patterns, suggesting morphological correlations that are consistent with a distal-proximal kinetic chain hypothesis. The experimental findings indicate that the performance benefit of multimodal attention fusion varied across distinct alignment tasks. Together, these findings demonstrate the potential of plantar three-dimensional scanning combined with multimodal, explainable deep learning as an accessible, non-invasive approach for screening alignment-related biomechanical risk factors, while warranting further external validation prior to clinical translation.

Subject of Research: Multimodal deep learning for screening lower-limb alignment using plantar 3D foot scans

Article Title: Development and internal validation of a multimodal deep learning model for screening-level lower-limb alignment using plantar 3D foot scans

Article References: Ni, S., Chen, Y., Wang, C., Zhou, D., Liu, J., Zhou, M., Leong, J., Hu, Y., Huang, J., Wang, J., & Liu, C. (2026). Development and internal validation of a multimodal deep learning model for screening-level lower-limb alignment using plantar 3D foot scans. PLOS Digital Health, 5(11), e0001802. https://doi.org/10.1371/journal.pdig.0001802

Image Credits: AI Generated

DOI: 10.1371/journal.pdig.0001802

Keywords: multimodal deep learning, plantar 3D scanning, lower-limb alignment, hindfoot alignment, frontal-plane knee alignment, arch type, knee osteoarthritis, plantar fasciitis, musculoskeletal loading, biomechanical risk factors, feature-wise attention, class activation mapping, vision transformer, geometric morphological features, kinetic chain hypothesis, cross-sectional study