Evolutionary AI Discovers Leaner Neural Networks by Treating Every Class as Its Own Goal

A team of researchers from Tel-Aviv University and Braude College of Engineering has introduced a fundamentally different way to design deep neural networks for classification tasks, one that treats the error on each class as a separate objective rather than folding everything into a single loss. The approach, described in an open-access paper in Neural Computing and Applications by Guy Mardiks, Adham Salih and Amiram Moshaiov, reframes multi-class classification as what the authors call a Class-based Many-objective Supervised Learning problem, or CMaO-SL. Instead of producing one network trained to minimize an aggregate error, the method yields an entire set of candidate classifiers, each striking a different balance between how well it recognizes individual classes. The result is not just a new algorithm but a new way of thinking about what it means for a classifier to be good.

The core insight behind the work is that classification errors across classes can conflict. A network that excels at distinguishing shirts from coats in the Fashion MNIST dataset might struggle with sneakers versus sandals, and the conventional practice of averaging errors into a single scalar loss hides these tensions entirely. By contrast, the CMaO-SL formulation defines one objective per class, so a ten-class problem becomes a ten-objective optimization problem. Because no single architecture can simultaneously minimize every class-specific error, the search targets a Pareto set: a collection of non-dominated solutions in which improving performance on one class necessarily comes at some cost to another. This trade-off information, normally invisible to practitioners, becomes an explicit output of the training process.

Solving such a problem is computationally daunting. Not only must the search evaluate many candidate architectures, but it must do so repeatedly to draw statistically meaningful conclusions about a stochastic process. The authors tackled this challenge with a two-pronged strategy combining transfer learning with evolutionary neural architecture search. Every network in their framework is split into two parts: a deep, pre-trained feature extractor kept completely frozen, and a shallower classification sub-network that is allowed to evolve. The feature extractor is derived from a ResNet-18 network pre-trained on ImageNet, with its final layers removed and its pooling adjusted to output 2,304 features. Because the backbone is never retrained or fine-tuned, the expensive part of the computation is paid only once, and the search can concentrate on the smaller, cheaper classification head.

The evolutionary search itself adapts MOEA/D, a well-established decomposition-based multiobjective evolutionary algorithm, to the architecture search setting. The user first defines a master architecture for the classification sub-network, consisting of fully connected layers with batch normalization and ReLU activations. Each individual in the evolving population is a binary mask vector that dictates which connections in the master architecture are pruned; where a mask entry is zero, the corresponding connection is removed and its weight fixed at zero. This pruning-as-search strategy generates a rich family of alternative architectures from a single blueprint. Each candidate is then trained with gradient descent, evaluated per class on a validation set, and assigned a performance vector whose components are one minus the F1 score for each class.

The evaluation procedure embeds an early-stopping mechanism to guard against overfitting. During each candidate’s five training epochs, the algorithm tracks a scalar validation error and retains the best-performing weight vector, ensuring that later epochs cannot degrade the network that gets assessed. The authors used two training loss functions in their experiments, cross-entropy and a weighted average of per-class binary cross-entropy aligned with each sub-problem’s reference direction, and two corresponding validation error functions. This produced five distinct search configurations, three evolutionary and two non-evolutionary, the latter serving as controls in which independent scalar searches replaced the coupled evolutionary process.

The experimental study spanned six benchmark datasets drawn from very different domains: Fashion MNIST, K-MNIST, CIFAR10, and three medical image collections from the MedMNIST suite, namely BloodMNIST, DermaMNIST and OCTMNIST. Each configuration was run fifteen times, allowing a rigorous statistical treatment that is rare in the Pareto-based architecture search literature. After confirming with a Shapiro-Wilks test that the results were not normally distributed, the authors compared configurations using medians and pairwise Mann-Whitney-Wilcoxon tests at a significance level of 0.05, alongside the IGD+ performance indicator, which measures how well an obtained front approximates a reference Pareto front.

The results were consistent across datasets and evaluation sets. Networks trained with cross-entropy loss produced better-approximated Pareto fronts than those trained with the weighted binary cross-entropy variant, and the evolutionary searches matched or beat their non-evolutionary counterparts on front quality while producing sets of networks with lower total complexity. Parallel coordinate plots of per-class F1 scores made the conflicting nature of the objectives vividly visible: the line connecting a classifier’s best performance on one class typically dipped on others. Crucially, when the authors picked the single best network from each evolutionary run’s set, measured by overall validation accuracy, those networks matched the accuracy of the unpruned master architecture while being substantially smaller, demonstrating that the many-objective formulation finds leaner models without sacrificing performance.

The benchmark comparisons are perhaps the most striking part of the study. When the networks discovered by the proposed method were plotted against published reference results for ResNet-18-based classifiers on the same datasets, no published result dominated the new networks on the combined axes of accuracy and complexity. The full ResNet-18 carries roughly 11.5 million parameters relevant to the comparison, whereas the evolved networks came in at around 2.5 million, meaning the reference architecture is about 360 percent more complex. That dramatic reduction in size cost less than 8 percent in accuracy, and on the DermaMNIST skin-lesion dataset the evolved classifiers actually surpassed the published benchmark. For a method that never fine-tuned its feature extractor and trained each candidate for only five epochs, this is a remarkable showing.

The practical implications extend beyond raw benchmark numbers. Because the output of the search is a set of classifiers with exposed trade-offs, users gain flexibility that single-model pipelines cannot offer. A deployment with tight computational budgets can select the smallest acceptable network from the set, while an application where a particular class matters most, say a rare disease category in a medical screening tool, can pick the member that favors that class. Alternatively, the trade-off structure can guide the construction of ensembles, an avenue the authors identify as a priority for future work. They also flag several open questions: whether fine-tuning the shared feature extractor would push accuracy higher, how the approach handles imbalanced data and noisy labels common in real-world problems, and how alternative pruning schemes, such as removing whole layers or blocks, might perform.

The study also fills a conceptual gap in the literature. Although Pareto-based multiobjective machine learning has existed for years, formal treatments of many-objective supervised learning, where the number of objectives exceeds three, have been scarce, and no prior work had combined such a formulation with neural architecture search and a transferred feature extractor. Most existing multiobjective NAS studies juggle only two or three objectives, typically accuracy, model size and inference cost, rather than one objective per class. By demonstrating a working pipeline at ten objectives, backed by fifteen-run statistical analysis per configuration, the Tel-Aviv and Braude team has shown that class-aware, many-objective architecture search is not only feasible but competitive. As automated machine learning matures, approaches like this suggest a future in which the question is no longer simply how accurate a network is, but which compromises it makes, and whether those are the compromises its users actually want.

Subject of Research: Class-based many-objective neural architecture search for deep classifiers using evolutionary optimization and transfer learning

Article Title: Class-based many-objective neural architecture search for deep classifiers

Article References: Mardiks, G., Salih, A., & Moshaiov, A. (2026). Class-based many-objective neural architecture search for deep classifiers. Neural Computing and Applications, 38(19), Article 800. https://doi.org/10.1007/s00521-026-12510-4

Image Credits: AI Generated

DOI: 10.1007/s00521-026-12510-4

Keywords: neural architecture search, many-objective optimization, Pareto front, evolutionary algorithms, transfer learning, deep learning, multi-class classification, MOEA/D, pruning, ResNet-18, AutoML, machine learning