Semantic Labels Guide Terrain-Specific Humanoid Behavior in Simulation

A semantic terrain label allowed a behavior-cloning policy to select distinct locomotion behaviors for a simulated Unitree H1 humanoid on flat ground and stairways. The study, published in Scientific Reports, reports that the unified policy retained 94.73% of the flat-ground reference-policy reward and 93.07% of the stairway reward in closed-loop simulator deployment. Counterfactual tests that changed the terrain labels also produced measurable changes in the predicted joint actions, indicating that semantic conditioning influenced the policy rather than merely accompanying its terrain input.

The researchers first trained two terrain-specific reference policies using Proximal Policy Optimization, one for flat-ground locomotion and the other for stairway locomotion. They then collected trajectories from those policies and used the demonstrations to train a Unified Multi-modal Behavior Cloning framework, abbreviated UMBC. This second stage was a supervised imitation approach: the unified model learned to reproduce the actions of the independently trained policies under different terrain conditions.

UMBC assigns separate encoders to proprioception, terrain geometry and semantic terrain identity. The proprioceptive encoder handles the robot’s internal-state information, while the geometry encoder represents features of the environment. A learned embedding represents the semantic terrain label, and that embedding explicitly conditions a shared policy head responsible for predicting joint actions. Across the test set, the proposed policy achieved an action mean absolute error of 0.016349 and a root mean square error of 0.025733; its best validation loss was 0.000367.

The counterfactual analysis found nonzero action changes throughout the independent test set. Mean L2 distances between predicted action vectors were 0.5019 when labels changed from flat ground to stairway and 0.5187 for the reverse substitution. Additional deployment experiments evaluated terrain transitions, transition delays and robustness under external perturbations in simulation. Ablation comparisons tested one-hot labels, removal of the terrain label and encoder-free concatenation, all of which produced lower terrain-specific performance than the proposed architecture. An adaptive terrain-dependent weighting scheme slightly improved stairway performance but reduced flat-ground performance. The results support semantic terrain conditioning as an explicit way to isolate terrain-specific behaviors within one policy, although the evidence is limited to simulator-based experiments on the Unitree H1 platform and does not establish performance on physical hardware or in unseen environments.

Subject of Research: Semantic conditioning for terrain-specific humanoid locomotion in simulation

Article Title: A semantic-conditioned multi-modal behavior cloning framework for terrain-specific behavior isolation in humanoid locomotion

Article References: Ahmed, A. S. F., Doss, A. S. A., & Hency, V. B. (2026). A semantic-conditioned multi-modal behavior cloning framework for terrain-specific behavior isolation in humanoid locomotion. Scientific Reports. https://doi.org/10.1038/s41598-026-75161-4

Image Credits: AI Generated

DOI: 10.1038/s41598-026-75161-4

Keywords: humanoid robotics, Unitree H1, semantic conditioning, behavior cloning, terrain adaptation, locomotion, multimodal learning, reinforcement learning, robot simulation, semantic-conditioned, multi-modal, behavior