A new robotic grasping system has demonstrated a striking ability to move between radically different robot hands without being retrained for each one. Called SpaHybGen, the framework combines neural perception with analytical motion planning to give machines a shared way of understanding where and how an object can be grasped. In experiments, a single model successfully supported seven robotic hands with different shapes and finger counts, ranging from two-finger designs to five-finger articulated hands. The system achieved grasping success rates between 94.3% and 98.0% in semi-cluttered environments, while also demonstrating dynamic grasping at 20 hertz in dense clutter and coordination between multiple robotic hands. The results, reported by X. Wang, L. M. Tam and Q. Xu in Nature Machine Intelligence, point toward a possible new architecture for general-purpose manipulation: one in which perception is learned once, while the final action is computed according to the hardware available.
Robotic grasping has long exposed a tension at the heart of artificial intelligence. Deep-learning policies can learn complex visual and motor patterns from data, but they often become tightly linked to the particular robot, camera arrangement and objects used during training. A policy trained for a parallel-jaw gripper may not transfer cleanly to a dexterous hand with several independently controlled fingers. Conversely, conventional analytical planners can reason about different robot geometries using physical models, but they typically require accurate object shapes, reliable collision information and carefully engineered assumptions. Real-world scenes rarely provide such ideal conditions. Objects may be partially hidden, depth sensors may contain gaps and noise, and clutter can make the exact geometry of a grasp difficult to determine. SpaHybGen is designed to combine the adaptability of learning with the hardware flexibility of analytical optimization.
The central idea is a learned spatial contact representation. Rather than asking a neural network to directly predict every motor command for a specific robot, the researchers train it to infer potential contact features from noisy depth observations. These features describe spatial regions where a robotic hand might establish useful contact with an object. They are not tied to the joints, link lengths or actuator arrangement of one particular machine. In effect, the network translates an incomplete three-dimensional observation into a universal contact map, identifying locations and spatial relationships that could support a stable grasp. This division is important because the same object-level contact information can be interpreted by many different hands. The representation becomes an interface between perception and action, allowing the visual system to remain unchanged while the downstream planner adapts to the robot’s physical structure.
After the contact features have been predicted, SpaHybGen uses a differentiable optimizer to calculate a grasp for a specific articulated hand. Differentiability means that the system can evaluate how changes in a hand’s pose, finger configuration or contact locations affect the quality of the proposed grasp and then use gradients to improve the solution. Instead of treating perception and planning as completely separate processes, the method allows the predicted contact information to guide a continuous search through the hand’s possible configurations. The optimizer can account for the geometry of the selected hand, including its joints and fingers, while seeking contacts that are physically plausible and stable. This hybrid arrangement avoids forcing the neural network to memorize the mechanics of every robot it may encounter. The learned component identifies where contact could be valuable; the model-based component determines how a particular machine can reach and use those contacts.
The reported experiments tested the framework across seven distinct robotic hands, providing a demanding examination of its claimed hardware independence. The hands differed in their mechanical designs and included configurations with between two and five fingers. Despite those differences, the researchers say the system operated without hardware-specific retraining. That result is significant because robot-learning systems commonly depend on demonstrations collected from the same platform that will later execute the task. Even small changes in finger placement, joint limits or wrist geometry can reduce performance when a policy has learned a direct mapping from images to actions. SpaHybGen instead transfers the shared contact representation and recomputes the grasp through the optimizer. The reported success range of 94.3% to 98.0% in semi-cluttered scenes suggests that the approach retained high reliability while moving across platforms with substantially different capabilities.
Clutter provides a particularly severe test for grasping systems because the robot must identify useful surfaces without confusing neighboring objects or selecting a pose that causes collisions. A depth camera may reveal only fragments of an object, leaving the network to infer viable contact regions from incomplete evidence. The researchers designed SpaHybGen to work directly with such noisy depth observations rather than relying on perfectly reconstructed object models. This is a practical distinction: in household, warehouse and industrial settings, objects are frequently occluded, reflective, irregularly placed or surrounded by other items. A contact representation can help the system focus on actionable geometry instead of requiring a complete visual description. By passing those predictions to a hand-aware optimizer, the framework can seek a grasp that respects both the visible scene and the physical limitations of the chosen robot.
The system also showed that the shared contact interface could support movement beyond static pick-and-place demonstrations. According to the study, SpaHybGen enabled dynamic grasping at 20 hertz in dense clutter. A 20-hertz control or planning rate corresponds to a new update approximately every 50 milliseconds, allowing the robot to react repeatedly as objects, viewpoints or hand positions change. This does not eliminate the challenges of fast manipulation, including sensing latency, actuator response and collision avoidance, but it indicates that the hybrid pipeline can operate at a rate relevant to responsive interaction. In dynamic conditions, the robot cannot depend on a single carefully prepared grasp pose. It must continually revise its estimate of promising contacts and update the hand configuration as the scene evolves. The ability to refresh decisions quickly is therefore as important as the quality of any individual grasp.
Another demonstration extended the framework to multi-hand coordination for complex tasks. Multiple hands introduce additional planning problems because each hand must not only establish stable contacts but also avoid interfering with the other robots or with the object’s intended motion. A system trained for one gripper would normally have little reason to understand these interactions, especially when the participating machines have different kinematic structures. SpaHybGen’s separation between contact perception and hardware-specific optimization offers a way to assign each hand a compatible grasp while preserving a common description of the object. The reported result suggests that a universal contact representation could become a building block for coordinated manipulation, in which several robots divide physical responsibilities without requiring every combination of hardware to be represented in the training data.
The work does not mean that robotic generalization has been solved. Stable grasping remains sensitive to friction, object weight, compliance, calibration errors and the accuracy of the estimated contacts. A contact feature that appears promising in depth data may fail if the surface is slippery or deformable, while an optimizer can produce a mathematically valid pose that is difficult for a real actuator to execute. The experiments also represent particular hardware, environments and task conditions, so broader testing will be needed to establish how well the framework transfers to unseen objects, rapidly moving targets and more severe occlusion. Even so, the researchers’ architecture addresses a central limitation of current robot learning by refusing to make the visual policy responsible for every mechanical detail. Its promise lies in separating what can be shared across robots from what must remain specific to each machine.
SpaHybGen’s broader importance is its proposed interface for reusable manipulation intelligence. If contact representations can reliably describe opportunities for physical interaction, they could allow perception models to be trained once and reused across a growing ecosystem of robotic hands, arms and mobile platforms. The analytical planner would then serve as a translation layer, converting shared spatial information into actions that obey the geometry and constraints of each device. This could reduce the enormous data burden associated with collecting demonstrations for every new robot and make advanced manipulation more accessible to researchers and manufacturers. Wang, Tam and Xu have open-sourced the code and models, creating an opportunity for independent groups to test the approach under different conditions. The reported results position learned contact representations as a possible bridge between general-purpose artificial intelligence and the messy, hardware-dependent reality of physical robotics.
Subject of Research: Universal robotic grasping through learned, hardware-agnostic spatial contact representations and differentiable optimization.
Article Title: Learning contact representations in real-world clutter for universal robotic grasping
Article References: Wang, X., Tam, L.M. & Xu, Q. Learning contact representations in real-world clutter for universal robotic grasping. Nat Mach Intell (2026). https://doi.org/10.1038/s42256-026-01292-y
Image Credits: AI Generated
DOI: https://doi.org/10.1038/s42256-026-01292-y
Keywords: robotic grasping, robot manipulation, contact representations, differentiable optimization, deep learning, cluttered environments, dexterous hands, zero-shot generalization, multi-hand coordination, artificial intelligence
Tags: contact learning for manipulation in real-world cluttercoordination of multiple robotic handsdynamic grasping at 20 Hz in dense cluttergeneral-purpose robotic manipulation architecturehigh success rates in cluttered environmentsneural perception combinedovercoming limitations of deep-learning policies in robotic graspingrobotic hand with different shape or number of fingersshared neural perception and analytical motion planning for universal graspingSpaHybGen framework for multi-robot hand adaptabilitytransferability of grasping policies across different robotic hardware
