Capacity Ratios Fail as a Safeguard Against Harmful Knowledge Distillation in Cross-Domain Pill Retrieval

Compact models running on edge devices often rely on knowledge distillation to close the accuracy gap that separates them from larger networks, but a new study reports that the rule most researchers use to pick a safe teacher is unreliable. In a large-scale evaluation of teacher-student pairings for cross-domain pill identification, harmful transfer appeared in 21.8 percent of 180 pairings on the ePillID benchmark and in 27.3 percent of 180 pairings on an independently derived CURE benchmark, each tested over three training seeds. The work, published in Complex & Intelligent Systems, was designed to test teacher selection rules at a scale that could falsify them rather than simply illustrate them with a handful of examples.

The central target of the analysis is the capacity ratio, the ratio of teacher to student model size that much of the distillation literature treats as a safeguard, and which the authors themselves had originally proposed. The study finds that this safeguard fails in three distinct ways. First, it discriminates weakly between harmful and beneficial pairings, achieving a screening area under the curve of only 0.66. Second, the threshold fitted to the capacity ratio does not transfer between benchmarks: the fitted values were 17.1 on one benchmark and 24.0 on the other, and neither matched the commonly asserted cutoff of 12. Third, when the cutoff of 12 is applied, no teacher qualifies for the two smallest students in the study, forbidding distillation precisely in the settings where compact models need it most.

Harmful transfer was not distributed evenly across architectures. On ePillID, 45.0 percent of pairings involving compact transformer students were labeled harmful, compared with 8.6 percent for convolutional neural network students. On CURE the pattern held, with 55.0 percent versus 11.4 percent. The authors note these architecture-level labels were assigned from single-seed runs, a limitation they flag explicitly, but the concentration in transformer students was consistent across both benchmarks.

The team then evaluated whether any training-free signal could rank candidate teachers better than a simple baseline: choosing the strongest teacher by its own benchmark accuracy. The candidates included four similarity metrics, fusions of those metrics, and five external transferability estimators. None outperformed the strongest-teacher baseline by more than the measured noise floor. In other words, for the question of which teacher will produce the best student, no inexpensive proxy in the test set added value over simply picking the most accurate model available.

A more modest question yielded a partial positive result: predicting whether a pairing will harm at all, rather than how much it will help. Here a correspondence-aware manifold divergence measure led on ePillID. Its advantage over the capacity ratio in screening AUC was not statistically resolved at the eleven student clusters examined, but the rule showed practical advantages. At a threshold fitted out of sample, and on the nine students where both rules produced a decision, the divergence-based screen admitted three harmful pairings against nine for the capacity ratio, while discarding 24 beneficial pairings against 39. It still led when the capacity ratio was given its own fitted threshold, and unlike the ratio it was defined for every student in the test set.

That partial success did not survive the change of dataset. The screen did not transfer visibly to CURE, where no signal the authors tested separated harmful pairings from safe ones. The study notes that the CURE experiment had power only against strong screens, meaning weaker discriminative signals could not have been detected there in any case. The overall tally of replacement candidates was sobering: of the alternatives evaluated, one result was modestly positive and the rest were negative.

The practical implications reach beyond pill identification. Cross-domain fine-grained retrieval, where models must match images of medications across lighting conditions, camera angles and packaging variations, is a representative case of the constraints facing deployed compact models. If the standard size-based heuristic for teacher selection both fails to predict harm and blocks distillation for the smallest students, practitioners are left without a reliable training-free rule, and the study suggests the strongest-teacher-by-accuracy baseline remains the most defensible default for ranking, with the divergence measure as a partial harm screen on benchmarks similar to ePillID.

The authors frame their contribution as a correction to their own earlier proposal, describing the capacity ratio’s apparent protective effect as an architectural mirage. The evidence base is unusually thorough for a negative-result study: 360 teacher-student pairings in total across two independently derived benchmarks and three seeds, with explicit reporting of noise floors, unresolved statistical comparisons and single-seed labeling limitations. The work does not offer a new universal safeguard; it documents that none of the tested training-free signals, including the most widely cited one, meets that standard, and it quantifies exactly where each falls short.

For the field of knowledge distillation, the message is that scale matters in evaluation. Rules that look sound in small illustrative experiments can fail screening tests, threshold transfer and coverage checks simultaneously when examined across hundreds of pairings. Until a validated alternative emerges, the study indicates that assumptions about safe teacher selection, particularly those based on model size ratios, should be treated with caution in any cross-domain setting where harmful transfer is a live risk.

Subject of Research: Failure of capacity ratio safeguards in teacher selection for knowledge distillation in cross-domain pill image retrieval

Article Title: The architectural mirage: capacity ratios fail as a safeguard against harmful knowledge distillation in cross-domain pill retrieval

Article References: Nhat, L. D., Tuan, D. L. T., & Hoang, V. T. (2026). The architectural mirage: capacity ratios fail as a safeguard against harmful knowledge distillation in cross-domain pill retrieval. Complex & Intelligent Systems. https://doi.org/10.1007/s40747-026-02531-w

Image Credits: AI Generated

DOI: 10.1007/s40747-026-02531-w

Keywords: knowledge distillation, capacity ratio, harmful transfer, pill identification, teacher selection, cross-domain retrieval, transferability estimation, compact models, ePillID, CURE benchmark, edge devices, screening AUC