Anyone who has ever tried to pull a specific book out from the middle of a crowded shelf knows that the hard part is not the reaching, it is the figuring out what sits on top of what. Robots face exactly the same problem, only worse, because they cannot rely on a lifetime of intuitive physics to guide them. A robotic arm staring into a bin of jumbled parts, or a service robot asked to retrieve a single item from a messy kitchen counter, must decide not only where the target is but which objects are covering it and in what order they must be removed. Getting this wrong means knocking over the entire pile, damaging goods, or halting an assembly line. A new study published in the International Journal of Intelligent Robotics and Applications by Yan Zha, Yuxuan Wang, Jianhua Wu and Zhenhua Xiong of Shanghai Jiao Tong University presents a computational approach designed to make exactly this kind of decision quickly, reliably, and in a way that keeps up with a changing scene.
The core contribution of the work is a neural architecture the authors call the Occlusion-Aware Pairwise Relational Network, or OAPRN. Rather than trying to understand an entire cluttered pile of objects in one monolithic computation, the network breaks the problem down into a sequence of pairwise judgments. For every pair of objects detected in the scene, the network answers a deceptively simple question: which one of the two is on top? This might sound like a crude simplification of the full three-dimensional reasoning problem, but the mathematics of stacking relationships is friendly to such decompositions. If object A occludes object B, and object B occludes object C, then a robot can infer that A must be dealt with before C, even if A and C never interact directly in the image. Stacking, in other words, is a transitive relation, and pairwise comparisons encode all the information needed to recover the full ordering.
This pairwise decomposition is not just an aesthetic choice; it carries real algorithmic consequences. When a relational network must process all objects and their interactions simultaneously, the size of its internal representation grows combinatorially with the number of objects, and training the network requires examples covering every conceivable clutter configuration. By contrast, a pairwise network sees a much narrower and more consistent learning problem: given two objects and their image evidence, predict the occlusion direction. Because each training example concerns just two objects, the diversity of scenes needed to teach the network the underlying visual cues is far smaller, and the learned features generalize more readily to piles with different numbers of items. Once individual pairwise predictions are made, the system assembles them into an overall relationship graph, a directed structure whose edges point from the covering object to the covered one, and from which the full stacking hierarchy of the scene can be read off.
The graph itself is where perception turns into action. With the relationship graph in hand, a robot asked to grasp a specific target object can trace the chains of occlusion leading to that object and identify exactly which items stand between the gripper and the goal. The grasping strategy then proceeds in a deliberately orderly fashion: objects that cover the target are removed first, beginning with those highest in the stack, and the target is grasped only once its immediate surroundings are clear. This sequential clearing avoids the classic failure mode of clutter manipulation, in which a greedy grasp of the target dislodges neighbors and buries it deeper. The authors describe how the robot can execute this plan quickly, because the computational cost of the pairwise network remains modest even as the pile grows, and because the graph reconstruction step operates on a small number of detected objects rather than on raw image data.
Efficiency is not merely a convenience in this setting; it is what allows the robot to cope with the fundamental non-determinism of manipulation. Every time a gripper removes an object, the remaining pile shifts, occlusions change, and previously hidden surfaces come into view. A system that needed several seconds to recompute the scene graph would be constantly planning against a world that no longer exists. Because OAPRN can rebuild the relationship graph rapidly from a fresh image, the robot can replan after every action, treating each removal as a new observation rather than a disruption to a fragile long-horizon plan. This closed-loop quality, in which perception, planning and action alternate at a pace matched to the physical dynamics of the clutter, is arguably the most practically important feature of the method, and it distinguishes the approach from heavier relational reasoning pipelines that produce accurate but stale scene interpretations.
The technical pipeline begins with standard object detection on the input image, for which the authors leverage modern detector tooling of the kind exemplified by the widely used YOLO family of models cited in their bibliography. Each detected object yields an image region and associated features, and these pairwise feature combinations are passed through the relational network to predict occlusion direction. The idea of detecting visual manipulation relationships in this way has a lineage in the robotics literature, notably in work on visual manipulation relationship networks and gated graph neural networks for robotic grasping, as well as graph-based reasoning for target-driven grasping in dense clutter. What the Shanghai Jiao Tong team adds is a focus on making this reasoning lightweight enough for real-time, iterative use, and on coupling it directly to a principled clearing strategy rather than to learned push-and-grasp policies that can be harder to interpret and verify.
The experimental section of the paper puts the method through its paces in cluttered scene trials designed to verify both the relationship detection and the resulting grasp behavior. The authors report that experiments were carried out to verify the proposed method, and the accompanying figures trace the pipeline from raw input images through pairwise relationship analysis, graph construction, and the executed sequence of removals culminating in the target grasp. A supplementary video accompanies the publication, offering a dynamic view of the system operating on physical clutter. The work was supported by the National Science and Technology Major Project under grant number 2024ZD1600204, and the authors note that no datasets were generated or analyzed during the study beyond those used in the reported experiments. Author contributions attribute the algorithm design to Yan Zha and Yuxuan Wang, the experimental work to Zha, Wang and Jianhua Wu, and the manuscript drafting to Zha, with all authors reviewing the final text.
Situating this work in the broader landscape of robotic grasping research clarifies why it matters. Grasp detection as a field has advanced enormously, from early deep learning approaches to grasp rectangles through modern six-degree-of-freedom grasp planners operating on point clouds, and systems such as AnyGrasp that perceive grasps robustly across space and time. Yet most of these methods answer the question of how to grasp an object, not whether it should be grasped yet. A parallel line of research, including work on target-oriented push-grasping synergies and hierarchical stacking relationship prediction, has tackled the order-of-operations problem, often at considerable computational expense or with limited adaptability to scene changes. Large vision-language-action models promise general manipulation competence, but their inference cost makes them poorly suited to the millisecond-level replanning loop that clutter demands. OAPRN occupies a pragmatic middle ground: a specialized, efficient relational reasoner that does one thing, determining what covers what, well enough to support fast and safe sequential manipulation.
The potential applications extend across both halves of the robotic world the authors name in their motivation. In industrial automation, random bin picking remains one of the highest-volume manipulation tasks in logistics and manufacturing, and any improvement in the speed or reliability with which a robot can extract a specific part from a tangled pile translates directly into throughput. In domestic service, where robots must fetch requested items from drawers, refrigerators, and countertops they have never seen arranged in advance, occlusion-aware reasoning is the difference between a helpful assistant and a hazard. The decomposition strategy also has an appealing quality for safety-critical settings: because the relationship graph is explicit and human-readable, an engineer can inspect exactly why the robot decided to remove one object before another, rather than trusting an opaque end-to-end policy. As robots move from structured cells into the genuinely messy environments where humans live and work, research of this kind, focused on making relational perception fast, transparent, and reactive, may prove as consequential as the grasp planners and foundation models that currently dominate the headlines.
Subject of Research: Occlusion-aware pairwise relational reasoning for efficient robotic grasping of target objects in cluttered scenes
Article Title: Efficient grasping of target objects in cluttered scenes
Article References: Zha, Y., Wang, Y., Wu, J., & Xiong, Z. (2026). Efficient grasping of target objects in cluttered scenes. International Journal of Intelligent Robotics and Applications. https://doi.org/10.1007/s41315-026-00595-y
Image Credits: AI Generated
DOI: 10.1007/s41315-026-00595-y
Keywords: robotic grasping, occlusion detection, relational networks, cluttered scenes, graph-based planning, robotic manipulation, object relationship detection, pairwise reasoning, computer vision, machine learning, industrial automation, Shanghai Jiao Tong University
Cite Scienmag News
APA MLA Chicago
Blake Davidson. (September 30, 2026). Teaching Robots to See Through Clutter with Pairwise Occlusion Reasoning. Scienmag. https://scienmag.com/teaching-robots-to-see-through-clutter-with-pairwise-occlusion-reasoning/
Blake Davidson. “Teaching Robots to See Through Clutter with Pairwise Occlusion Reasoning.” Scienmag, 30 September 2026, https://scienmag.com/teaching-robots-to-see-through-clutter-with-pairwise-occlusion-reasoning/. Accessed 30 September 2026.
Blake Davidson. “Teaching Robots to See Through Clutter with Pairwise Occlusion Reasoning.” Scienmag. September 30, 2026. https://scienmag.com/teaching-robots-to-see-through-clutter-with-pairwise-occlusion-reasoning/
Copy citation Download RIS
Tags: cluttered scenescomputer visiongraph-based planningindustrial automationMachine learningobject relationship detectionocclusion detectionpairwise reasoningrelational networksrobotic graspingrobotic manipulationShanghai Jiao Tong University
