Hybrid Classifiers Match End to End Deep Learning on Steel Defect Benchmarks

Hybrid machine learning classifiers can match the performance of end-to-end deep neural networks on highly separable metal inspection tasks, though end-to-end architectures demonstrate greater resilience as visual complexity and class imbalance increase. Researchers evaluated a comprehensive framework for steel surface defect classification to determine how different algorithmic architectures behave under controlled, augmented, and external benchmarking conditions. The investigation systematically compared end-to-end convolutional neural network backbones against hybrid configurations where extracted visual representations were classified using traditional machine learning algorithms.

The experimental framework incorporated six convolutional neural network backbones: MobileNet-V2, ResNet-50, DenseNet-121, EfficientNet-B0, U-Net, and LSDNet. Each backbone was evaluated in a native end-to-end deep learning configuration and also paired with four distinct machine learning classification heads, namely Logistic Regression, Support Vector Machine, Decision Tree, and K-Nearest Neighbors. In total, the investigation evaluated thirty unique backbone-head combinations, enabling direct comparisons between pure deep learning and hybrid classification strategies across standardized defect detection pipelines.

To evaluate model robustness across varying operational conditions, the researchers conducted evaluations across three distinct image datasets. The controlled baseline utilized the original NEU dataset, comprising 1,800 images distributed across six defect classes. To examine sensitivity to training volume and artificial variation, the authors utilized a source-disjoint augmented NEU dataset expanded to 3,600 images. Finally, to rigorously test generalization beyond original training distributions, the GC10-DET dataset—consisting of 2,312 images spanning ten distinct defect classes—served as an independent external benchmark.

Performance evaluation followed an experimental protocol relying on five-fold cross-validation and out-of-fold predictions. The authors tracked multiple performance metrics to avoid distortion from class distributions, measuring overall accuracy, balanced accuracy, macro precision, macro recall, and macro F1-score. Additional statistical assessments included the Matthews Correlation Coefficient, Cohen’s Kappa, and receiver operating characteristic area under the curve. To ensure empirical reliability, the researchers computed bootstrap confidence intervals and performed formal statistical significance tests across the competing configurations.

On the original NEU dataset, experimental results demonstrated clear benchmark saturation, with eight of the thirty evaluated backbone-head combinations achieving perfect out-of-fold accuracy. Furthermore, every tested configuration exceeded an accuracy threshold of 0.9867 on this baseline. When evaluated on the source-disjoint augmented NEU dataset, performance gains were predominantly confined to weaker model configurations, where the largest observed improvement occurred for the LSDNet backbone paired with a Decision Tree classifier head, yielding an accuracy increase of 0.0089.

In contrast to the saturated performance observed on the NEU dataset, the external GC10-DET benchmark generated substantially greater performance variation across configurations. On this ten-class benchmark, DenseNet-121 paired with a Logistic Regression head achieved the highest overall accuracy of 0.8984. However, DenseNet-121 utilizing the native end-to-end deep learning classification head attained the highest macro F1-score of 0.8681 and the highest balanced accuracy of 0.8609, illustrating that classifier head disparities become significantly more pronounced under challenging distributional conditions.

Statistical analyses confirmed that differences between classifier heads were far more substantial on the GC10-DET dataset than on the original NEU benchmark. These findings indicate that while hybrid classifiers effectively match end-to-end deep learning on highly separable benchmarks, integrated end-to-end representations become increasingly vital when systems encounter visual overlap, class imbalance, and elevated distributional complexity. To investigate model decision boundaries, the researchers applied Grad-CAM, Grad-CAM++, LIME, and SHAP, which provided complementary qualitative explanations highlighting the specific image regions driving defect predictions.

Subject of Research: Hybrid deep learning and machine learning framework for steel surface defect classification

Article Title: Classification of metal surface defects using a hybrid deep learning and machine learning approach

Article References: Kilci, O., Koklu, M., Alkanan, M., & Gulzar, Y. (2026). Classification of metal surface defects using a hybrid deep learning and machine learning approach. Journal of Big Data. https://doi.org/10.1186/s40537-026-01590-7

Image Credits: AI Generated

DOI: 10.1186/s40537-026-01590-7

Keywords: steel surface defect classification, hybrid deep learning, convolutional neural networks, MobileNet-V2, ResNet-50, DenseNet-121, EfficientNet-B0, U-Net, LSDNet, NEU dataset, GC10-DET dataset, five-fold cross-validation, Matthews Correlation Coefficient, Cohen's Kappa, Grad-CAM, LIME, SHAP