AI Judges Its Own Fashion Images Using a Classic Theory of How We See Clothes

AI-generated fashion images are flooding e-commerce catalogs, but nobody has been entirely sure how to check whether the digital garment still matches the real one. A new study in Fashion and Textiles proposes an answer: let vision-language models, or VLMs, grade the images themselves, guided by a fashion theory from 1987.

The problem the researchers set out to solve is well known in generative AI circles. Standard quantitative metrics such as the Structural Similarity Index Measure, Learned Perceptual Image Patch Similarity, and the Fréchet Inception Distance were built for general-purpose image evaluation. They measure pixel-level structure, learned perceptual distance, or statistical similarity between image distributions, but they were never designed to notice that a logo has vanished, a pocket has migrated, or a sleeve has changed shape. Human evaluation catches these details, but it is expensive, slow, subjective, and hard to reproduce consistently across raters.

To bridge that gap, the team grounded their evaluation framework in DeLong’s framework for visual analysis of dress, which holds that people perceive garments through five interdependent dimensions: texture, color, point, line, and shape. Drawing on prior fashion image synthesis studies that had examined attributes like logos, buttons, folds, transparency, and overall length without a shared theory, the researchers mapped empirically identified features onto the five dimensions. Texture-related attributes include motif alignment, motif size, transparency, and material; color covers hue, value, and intensity; point covers accessories, logos, and pockets; line covers seams and folds; and shape covers hem, overall length, fit, proportion, collar, neckline, and sleeve.

The evaluation procedure itself relies on carefully engineered prompts. Each VLM received a zero-shot prompt with four components: a role framing the model as a fashion image expert, a task specifying that only the target garment should be compared while background, pose, and styling items are ignored, the complete list of evaluation criteria with definitions, and a structured output format. For every applicable attribute, the model assigned a binary label of consistent or inconsistent, along with a brief chain-of-thought explanation. Because not every attribute applies to every garment type, only category-relevant attributes were assessed.

The test case was ghost mannequin photography, an increasingly common commercial application in which image generation models render garments with a natural fit and shape as if worn, but without any visible model or mannequin. The researchers collected 200 original images of worn garments by web crawling, 100 each from the Korean platform W-concept and the American platform Shopbop, spanning tops, outerwear, dresses, and bottoms. Three image generation models, Runway Gen-4, Gemini 2.5 Flash Image, and GPT-image-1-mini, produced ghost mannequin versions of each image under controlled conditions, yielding 600 generated images for validation.

Three state-of-the-art VLMs performed the automated evaluation: OpenAI’s GPT-5.4, Anthropic’s Claude Opus 4.6, and Google’s Gemini 3.1 Pro. Their outputs were benchmarked against two reference standards. The first was semantic similarity computed with FashionCLIP, a fashion-adapted CLIP variant, and SigLIP 2, a general-purpose vision-language encoder, using cosine similarity between embeddings of cropped garment regions. The second was human evaluation by three fashion industry professionals with at least ten years of experience each, who independently assessed all 600 image pairs, producing 11,589 attribute-level assessments per evaluator. Inter-rater reliability was high, with an intraclass correlation coefficient of 0.96.

The results split cleanly. Human evaluation found Gemini 2.5 Flash Image generated the most consistent garments, with mean consistency scores of 0.95 on both datasets, while Runway Gen-4 scored around 0.79 to 0.80 and GPT-image-1-mini around 0.67 to 0.69. The VLM-based evaluation reproduced this ordering, ranking the same three models in the same sequence, and Pearson correlations between human and VLM assessments ranged from 0.43 to 0.61. The quantitative metrics, by contrast, produced narrowly distributed similarity scores that failed to distinguish the models and even reversed the ordering of Runway Gen-4 and GPT-image-1-mini, correlating only weakly with human judgment at 0.01 to 0.45. In one illustrative example, GPT-image-1-mini earned a higher semantic similarity than Runway Gen-4 despite visible inconsistencies in sleeves, accessories, and pockets that both human and VLM evaluators caught.

But the attribute-level analysis revealed a systematic blind spot in the AI judges themselves. The researchers computed, for each attribute, the ratio of inconsistencies identified by the VLMs to those identified by human evaluators. The highest ratios clustered in the color and texture dimensions: intensity at 4.84, value at 3.16, motif size at 2.96, hue at 1.96, and motif alignment at 1.92. The lowest ratios fell in the shape and line dimensions: overall length at 0.75, sleeve at 0.96, fit at 1.17, proportion at 1.19, and seam at 1.20. At the dimension level, color showed the highest ratio at 2.93, followed by texture, point, shape, and line.

This asymmetry was shared across all three VLMs but varied in magnitude. GPT-5.4 flagged more inconsistencies than humans in every one of the five dimensions and all 20 attributes, while Claude Opus 4.6 fell below human counts in nine attributes and Gemini 3.1 Pro in five, all belonging to shape and line. The authors connect this pattern to a known property of vision architectures: convolutional networks and the vision encoders inside VLMs are biased toward texture over shape, unlike humans, who rely predominantly on shape for object recognition. Because VLMs typically divide images into grids of small patches processed as token sequences, surface properties like color and texture can be captured within single patches, while structural properties like shape and line span multiple patches and require integration across them.

The authors caution that these patterns reflect the current state of VLM-based evaluation rather than fixed properties of the technology. Prompt design, input format, and attribute definitions were held constant across models for comparability, so different prompting strategies might shift the results, and future VLM generations may become more sensitive to structural properties. The dataset was relatively limited, the human panel small, and the framework currently yields binary attribute judgments rather than a weighted overall score. Still, the method offers fashion e-commerce a scalable, theoretically grounded quality control tool that complements both blunt quantitative metrics and costly human inspection, and the framework can extend to fashion image generation tasks well beyond ghost mannequin photography.

Subject of Research: A vision-language model framework for evaluating garment consistency in AI-generated fashion images

Article Title: A VLM-based framework for evaluating garment consistency in AI-generated images based on DeLong’s theory

Article References: Park, H., Choi, B., & Choi, Y. (2026). A VLM-based framework for evaluating garment consistency in AI-generated images based on DeLong’s theory. Fashion and Textiles, 13(1), Article 26. https://doi.org/10.1186/s40691-026-00475-w

Image Credits: AI Generated

DOI: 10.1186/s40691-026-00475-w

Keywords: vision-language models, AI-generated images, garment consistency, ghost mannequin photography, DeLong visual definers, fashion e-commerce, image evaluation metrics, FashionCLIP, human evaluation, prompt engineering, VLM-based, framework