Contrastive Visual Framework Separates Competition, Suppression, and Missing Evidence in Neural Networks

In visual recognition, standard attribution methods for convolutional neural networks have long concentrated on answering a single fundamental question: which pixel regions prompted a classifier to predict a specific category? Traditional explainable artificial intelligence techniques, including Class Activation Mapping, Gradient-weighted Class Activation Mapping, Integrated Gradients, and Layer-wise Relevance Propagation, generate heatmaps highlighting evidence that favors a selected class. However, in practical deployment scenarios, such as clinical radiology or high-stakes inspection, practitioners frequently require answers to the complementary contrastive inquiry: why did the network predict a particular target class rather than a specific competing foil? While existing contrastive methods compute a direct mathematical difference between class scores, they typically merge opposing influences into an entangled aggregate signal. To address this structural limitation, researchers Adithya JK Annamalai and L. RamaParvathy from the School of Computer Science and Engineering at Vellore Institute of Technology have introduced Suppression-Absence-Competition eXplanations, designated as SACX, a framework that decomposes the contrastive margin into three spatially aligned visual components.

The mathematical foundation of SACX centers on examining the contrastive logit margin between the predicted class and a competing foil class. Standard attribution frameworks discard negative gradient contributions by applying a Rectified Linear Unit operation immediately after calculating the channel-weighted sum across feature activations in the final convolutional block. SACX instead preserves the complete signed weighted sum, decomposing the signal into distinct positive and negative operational streams. Generating this comprehensive representation requires only one backward pass for the winning predicted class and one backward pass for the competing foil class, making the computational overhead minimal. From these directional gradients, the framework derives three separate spatial heatmaps: a competition map that isolates positive visual evidence supporting the winning decision, a suppression map capturing negative attribution that actively penalizes the competing foil, and a prototype-derived absence map that pinpoints regions where visual cues characteristic of the foil class are conspicuously missing from the input image.

Within this tripartite decomposition, each visual heatmap addresses a structurally distinct reason for the classifier’s comparative choice. The competition component is mathematically defined as the normalized positive attribution of the winning class, which directly corresponds to the standard winning-class activation map reframed as an active competitor against the foil. Conversely, the suppression component isolates the normalized negative gradient contributions directed toward the foil category, identifying specific image regions whose activation correlates with a depression of the foil class logit score. Crucially, the researchers emphasize that this suppression map represents negative attribution under linear gradient approximation rather than an unconstrained guarantee of causal suppression, demonstrating that its behavioral impact varies under tight spatial constraints. Finally, because gradient-based maps cannot inherently signal the physical absence of expected visual patterns, SACX formulates its third component through an external prototype similarity bank, enabling the localization of expected foil features that never appeared within the input image.

To rigorously contextualize the aggregate contrastive signal, the authors demonstrate that an overall contrastive map synthesized from their formulation is mathematically equivalent to Contrastive Grad-CAM. Because the gradient weighting operations operate linearly over the logit difference between the winner and foil classes, the linear combination of channel weights preserves the identical differential representation. Consequently, the research does not claim novelty in producing an aggregate contrastive map, but rather in achieving a structured, interpretable decomposition of that established signal into distinct competition, suppression, and absence indicators. Whereas prior unified contrastive formulations yield a single difference map that fails to reveal whether an highlighted area represents active winner support, visible suppression of the competing label, or the total absence of foil attributes, SACX cleanly disentangles these mechanisms, allowing practitioners to inspect each behavioral dimension independently across challenging fine-grained visual classification tasks.

To generate the missing-evidence absence map without requiring expensive generative models or architectural modifications, SACX introduces a dual-tier PrototypeBank. When model training data is entirely unavailable during post-hoc inference, the framework activates a Tier-1 protocol that repurposes the normalized weight vector from the classifier final linear fully connected layer as a holistic class prototype. When training images are accessible, the system utilizes a Tier-2 clustering protocol. Under this preferred configuration, the researchers collect patch-level activation vectors from training exemplars, apply L2 normalization, and execute K-means clustering to establish sixteen representative centroid prototypes per class. By calculating the local cosine similarity between spatial feature tokens and these stored class prototypes against a calibrated twenty-fifth percentile threshold, the absence map flags regions exhibiting severe feature deficits. Weighting this absence map by the winning class spatial prior effectively shifts the diagnostic focus from unconstrained absence to the comparative question of why the winner was chosen over the foil.

Evaluating the empirical validity of contrastive explanations required the authors to re-evaluate existing faithfulness benchmarks, revealing a major confound in standard feature-injection tests. Traditionally, feature-injection area under the curve metrics measure whether inserting class-specific features into an image increases the target logit score. However, control experiments using non-informative spatial baselines revealed that inserting foil-class prototypes increases the foil logit score almost uniformly, regardless of whether features are placed into top-ranked absence locations, uniform random coordinates, or inverted spatial maps. Because aggregate feature-injection scores are dominated by the mere presence of prototype vectors rather than spatial localization quality, the authors developed a spatially controlled metric called Localization Fidelity under Injection. Under this direction-aware metric, prototype vectors are injected into a fixed budget of spatial cells, specifically comparing the logit gains attained by top-ranked absence coordinates against identical budgets of random, low-ranked, and spatially shuffled feature cells.

The experimental evaluation of SACX spanned fine-tuned ResNet-50 models, DenseNet-121 architectures, and VGG-19 networks evaluated across the fine-grained CUB-200-2011 bird dataset and PASCAL Visual Object Classes 2007. On fine-tuned ResNet-50 classifiers classifying bird species, injecting foil prototypes into the top twenty percent of absence coordinates achieved an average logit improvement of plus 0.407, with a ninety-five percent confidence interval spanning 0.391 to 0.421, substantially outperforming uniform random placement. On the multi-label PASCAL VOC benchmark, the performance advantage was even more pronounced, yielding a top-versus-random logit gap of plus 1.293 with a confidence interval between 1.229 and 1.358. In sharp contrast, evaluating an off-the-shelf ImageNet-pretrained ResNet-50 backbone without fine-tuning produced a near-zero localization gap of plus 0.034, confirming that the reliable localization of missing visual evidence requires backbones whose intermediate representations have been meaningfully aligned to task-specific spatial structures.

Beyond conventional convolutional backbones, the researchers investigated whether the SACX decomposition principles transfer to modern foundation models by evaluating a frozen DINOv2 Vision Transformer with a linear classification probe. On this self-supervised transformer architecture, inserting prototype vectors into the spatial patch grid proved universally destructive to the model internal representations, causing all variations of feature injection to decrease the raw foil logit score. Remarkably, however, the spatial fidelity metric revealed that the top-ranked absence locations remained the least disruptive sites across the grid. Evaluating a twenty percent spatial budget on the frozen DINOv2 model produced a positive top-versus-random logit advantage of plus 0.684, demonstrating that the structural ranking of missing evidence remains intact under frozen self-supervised representations even when the absolute numerical logit response requires task-specific supervised training to produce positive gains.

Ablation investigations confirmed the structural stability of the individual explanation components across diverse parameter configurations and architectural variants. Testing the absence map across different prototype cluster counts revealed that classification gains peaked at sixteen centroids, whereas higher prototype densities led to over-specialization and diminished spatial generalization. Examining the spatial focusing prior demonstrated that while multiplying the absence deficit by the winning class prior sharpens visual display and prevents diffuseness, the underlying spatial ordering is driven intrinsically by low foil similarity rather than high winning-class gradients, successfully resolving potential concerns regarding methodological circularity. Furthermore, substituting alternative base attribution backbones, including Grad-CAM++ and XGrad-CAM, maintained consistent missing-evidence localization gaps, while XGrad-CAM substantially improved the suppression-removal metric by providing better-normalized gradient coefficients across multi-scale spatial budgets.

To evaluate whether the mathematical decomposition translates into tangible visual utility for human interpreters, the authors conducted a rigorous perceptual forced-choice evaluation involving forty-five human participants across twenty balanced image trials. Observers evaluated side-by-side comparisons contrasting the SACX missing-evidence heatmap against standard foil Grad-CAM visualizations on images containing overlapping object categories. Participants were asked to identify which visual explanation better highlighted the specific missing features necessary to transform the image toward the competing foil category. The human observers demonstrated a statistically significant preference for the SACX absence maps, selecting them in 63.9 percent of trials, significantly surpassing the fifty percent random chance baseline with a p-value below 0.001. Additionally, bounding-box evaluations on false-negative classifications confirmed that the absence map concentrated 60.7 percent of its spatial energy directly within the ground-truth annotations.

The study clearly outlines several technical boundaries and open engineering challenges associated with the current implementation. Because the framework relies fundamentally on gradient signals, it remains susceptible to classical gradient saturation effects, and constructing high-fidelity Tier-2 prototypes requires adequate collections of representative class training instances. Furthermore, while the prototype-derived absence map successfully transfers to frozen vision transformers, the gradient-based suppression component exhibited reduced stability across linear probes on self-supervised representations, indicating that gradient flow across a single linear classification layer differs markedly from fully fine-tuned convolutional hierarchies. Despite these limitations, SACX establishes a rigorous theoretical and empirical foundation for contrastive explainability, moving beyond monolithic saliency heatmaps to deliver structured, direction-aware diagnostic insights into the internal mechanics of deep visual recognition systems.

Subject of Research: Structured contrastive visual explainability framework for deep image classifiers

Article Title: SACX: Suppression–Absence–Competition eXplanations for the why-not question in visual recognition

Article References: Annamalai, A. J., & RamaParvathy, L. (2026). SACX: Suppression–Absence–Competition eXplanations for the why-not question in visual recognition. Frontiers in Artificial Intelligence, 9, Article 1933918. https://doi.org/10.3389/frai.2026.1933918

Image Credits: AI Generated

DOI: 10.3389/frai.2026.1933918

Keywords: Explainable Artificial Intelligence, Contrastive Explanations, Convolutional Neural Networks, Grad-CAM, Suppression-Absence-Competition, Visual Recognition, Feature Injection, Localization Fidelity, PrototypeBank, DINOv2, Vision Transformers, ResNet-50, CUB-200-2011, PASCAL VOC 2007, Attribution Methods, Faithfulness Metrics, Saliency Maps