In a development that could reshape how machines search through billions of images, a team of Moroccan researchers has fused two very different intellectual traditions — Cold War-era game theory and modern deep learning — into a single image retrieval system that treats competing visual features as rival players in a strategic negotiation. The study, published in Multimedia Tools and Applications, describes how color, texture, and shape descriptors can be pitted against one another in a non-cooperative game, with the search refining itself iteratively until no player can improve the outcome. The result is a content-based image retrieval (CBIR) framework that reaches a Nash equilibrium before it returns its final answer, and it outperforms approaches that rely on any single type of visual evidence.
The research, led by Meriem Mansouhi and Samira El Moumen of the Computer Science and Systems Laboratory at Hassan II University of Casablanca, together with Noureddine Moussaid of the university’s Mathematics, Computer Science and Applications Laboratory in Mohammedia, tackles a stubborn problem in computer vision. Traditional CBIR systems rely on handcrafted features such as color histograms, shape signatures, and texture statistics to find images that look like a query image. Each of these features captures a different slice of visual identity, and none of them is sufficient on its own. A red sports car and a red rose may share a color histogram, while a car photographed at dusk may share nothing with the same car in daylight except its shape. Systems that naively blend features often produce results that satisfy no criterion particularly well — the visual equivalent of a committee compromise.
Mansouhi and her colleagues approached the problem differently. Instead of fusing features with fixed weights, they reformulated retrieval as a multi-criteria optimization problem and solved it with Nash game theory, the mathematical framework famously introduced by John Nash in the 1950s to describe strategic interactions among rational competitors. In the new framework, each feature extraction method is modeled as an independent player in a non-cooperative game. Every player proposes the cluster of images that it judges most visually similar to the query, based on its own representation of the visual world. The game then proceeds through an iterative negotiation: at each step, the intersection of all the proposed similar-image sets becomes the new working dataset, and the process is reapplied to that reduced set. The search narrows like a funnel, round after round, until a Nash equilibrium is reached — a state in which no player can unilaterally improve the retrieval result by changing its proposal.
The technical foundation of the game is a carefully engineered hybrid descriptor set. On the classical side, the system extracts color histograms, a technique whose lineage in image retrieval stretches back to Swain and Ballard’s seminal 1991 work on color indexing. Color histograms encode the distribution of pixel intensities across color channels in a way that is compact, fast to compute, and robust to small changes in viewpoint and lighting. On the deep learning side, the team adapted a customized variant of the VGG16 convolutional neural network, the sixteen-layer architecture introduced by Simonyan and Zisserman that became a backbone of modern computer vision. The researchers stripped out the network’s fully connected layers and applied Global Average Pooling to the convolutional outputs, a move that collapses spatial feature maps into compact, fixed-length vectors. This transformation yields descriptors that are simultaneously smaller, more generalizable, and directly comparable across images, while preserving the rich texture and shape information encoded in the convolutional layers.
Once the deep descriptors are extracted, the framework applies unsupervised k-means clustering to organize the image collection into groups within the feature space. Clustering serves a dual purpose: it accelerates retrieval by restricting the search space, and it produces more discriminative image groupings, so that when each game player proposes its candidate cluster, the proposal is meaningful rather than arbitrary. The color histogram player, the deep texture player, and the deep shape player each evaluate the clusters according to their own internal similarity metric, and the intersection mechanism enforces a form of consensus that no single feature could achieve alone. The design echoes the researchers’ earlier work, including a 2021 study by Elmoumen, Moussaid, and Aboulaich that applied Nash equilibrium concepts to image retrieval, but the new system is the first to combine that game-theoretic engine with deep convolutional features and clustering.
To test the framework rigorously, the team evaluated it on two demanding and deliberately contrasting benchmarks. The Oxford5K dataset, originally collected from Flickr images, contains thousands of photographs of Oxford landmarks and is a standard testbed for object and building retrieval, where lighting, viewpoint, and occlusion vary wildly. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of pigmented skin lesions, presents an entirely different challenge: medical images where subtle texture differences can separate benign lesions from malignant ones. The dual evaluation is significant because it demonstrates that the game-theoretic architecture is not tuned to one domain. A system that performs well on both landmark photography and dermatoscopy suggests a general-purpose retrieval mechanism rather than a domain-specific trick.
The implications for medicine are particularly striking. Content-based medical image retrieval has become an active frontier in clinical AI, with recent studies evaluating pre-trained convolutional networks and foundation models as feature extractors for diagnostic support. A retrieval system that can weigh texture cues captured by deep networks against shape and color evidence — and resolve conflicts between them through formal negotiation — could help clinicians find similar past cases for a difficult diagnosis, effectively letting a hospital’s entire imaging archive serve as a decision-support tool. The HAM10000 results suggest the framework is a credible candidate for that role, though the authors note that both datasets are used strictly for non-commercial research in accordance with their respective terms of use.
What makes the approach conceptually elegant is the way it reframes a familiar engineering problem in the language of economics. Multi-criteria optimization has long been handled with weighted sums, Pareto fronts, or heuristic fusion rules, all of which require a designer to decide in advance how much each criterion matters. The Nash game formulation removes that decision. Each feature player acts rationally in its own interest, and the equilibrium that emerges is a solution that no criterion can dispute. The iterative intersection process also gives the system a natural efficiency benefit: as rounds progress, the candidate set shrinks, so later rounds become faster even as they become more precise. The game, in effect, prunes its own search space.
The work also connects to a broader resurgence of interest in Nash-based methods for image processing. The same research group has previously explored a fractional-order derivative variational framework for blind image deconvolution using a Nash game strategy, and related work has applied game theory to machine learning and similar-image techniques. Meanwhile, the deep learning component sits within a rapidly evolving landscape that includes residual networks, contrastive vision-language models such as CLIP, and newer foundation encoders like SigLIP 2 and DINOv3. The Moroccan team’s contribution is not to compete with those giant models on raw representational power, but to show that a modest, carefully adapted VGG16 variant, combined with classical color descriptors and disciplined game-theoretic aggregation, can deliver competitive retrieval without massive compute.
The study received no external funding, and the authors declare no competing financial interests. Mansouhi led the literature review, methodology design, implementation, experiments, data analysis, and drafting, while El Moumen and Moussaid jointly conceived the research idea, provided methodological supervision, and critically revised the manuscript. The datasets — Oxford5K from academic benchmark repositories and Kaggle, and HAM10000 from a public Kaggle repository — are freely available to other researchers, which should make replication and extension straightforward for labs working on retrieval, medical imaging, or applied game theory.
As image collections worldwide swell into the billions and AI systems are increasingly asked to find, rank, and compare visual content, the question of how to combine competing notions of similarity grows more urgent. Search engines, medical archives, e-commerce platforms, and digital libraries all face the same fundamental tension: color says one thing, texture another, shape a third, and someone — or something — must arbitrate. By handing the arbitration to Nash’s equilibrium concept, the Casablanca team has offered a mathematically principled answer, one in which the final retrieval result is not a compromise imposed from above but a stability that emerges from below. If the approach scales as the benchmarks suggest, the next generation of image search may be quietly running a negotiation every time you press enter — and ending it only when every player agrees that no one can do better.
Subject of Research: Content-based image retrieval combining convolutional neural network features and game theory via Nash equilibrium optimization
Subject of Research: Technology and Engineering
Article Title: Game theory and CNN for similar images retrieval
Article References: Mansouhi, M., El Moumen, S., & Moussaid, N. (2026). Game theory and CNN for similar images retrieval. Multimedia Tools and Applications, 85(9), Article 730. https://doi.org/10.1007/s11042-026-21829-y
Image Credits: AI Generated
DOI: 10.1007/s11042-026-21829-y
Keywords: Game theory, Machine learning, Convolutional neural network, Image retrieval, Color descriptors, Shape descriptors, Texture descriptors
Cite Scienmag News
APA MLA Chicago
Blake Davidson. (September 3, 2026). CNN-Based Game Theory Approach Improves Similar Image Retrieval. Scienmag. https://scienmag.com/cnn-based-game-theory-approach-improves-similar-image-retrieval/
Blake Davidson. “CNN-Based Game Theory Approach Improves Similar Image Retrieval.” Scienmag, 3 September 2026, https://scienmag.com/cnn-based-game-theory-approach-improves-similar-image-retrieval/. Accessed 3 September 2026.
Blake Davidson. “CNN-Based Game Theory Approach Improves Similar Image Retrieval.” Scienmag. September 3, 2026. https://scienmag.com/cnn-based-game-theory-approach-improves-similar-image-retrieval/
Copy citation Download RIS
Tags: CNN-based image similarity searchcombining color texture shape featurescompetitive feature modelingcontent-based image retrieval (CBIR) systemscontent-based image retrieval systemsdeep learning and traditional features in multimedia searchdeep learning for image searchdeep learning in computer visionfusion of game theory and neural networksgame theory in image searchgame theory-based image retrievalimage retrievalinnovative content-based image search methodsiterative image search refinementmachine learning for image retrievalmodern image retrieval techniquesmodern image search algorithmsmulti-feature image matchingmulti-feature image similarity assessmentNash equilibrium in image matchingNash equilibrium in image processingnon-cooperative game models in image retrievalrival visual features in computer visionstrategic negotiation in visual feature analysis
