Hybrid CNN-Transformer AI Classifies Melanoma with 98.55% Accuracy on ISIC 2019

An artificial intelligence model that combines two complementary deep learning architectures has achieved strong performance on a large benchmark task for recognizing melanoma in dermoscopic images. The framework, called LHCT-Net, was developed by researchers at Vellore Institute of Technology in India and evaluated on the 25,331-image ISIC 2019 training collection. The work, published in Frontiers in Artificial Intelligence, reports 98.55% accuracy on a locked test set for the specific task of distinguishing melanoma from all other diagnostic categories in the dataset.

The model’s architecture reflects a growing trend in medical image analysis: pairing convolutional neural networks, which excel at capturing local visual detail, with Transformers, which model long-range relationships across an image. LHCT-Net begins with a fixed U-Net-based lesion localization step, followed by mask-guided lesion augmentation applied only during training. A boundary-aware EfficientNet-B0 branch then extracts features of local lesion morphology, while a Swin Transformer branch captures global context. The two streams are merged through attention-based local-global fusion and passed through a Lightweight Feature Optimization Block, with qualitative Grad-CAM++ visualizations used to inspect what the network attends to.

The evaluation used a fixed 70:15:15 stratified image-level holdout split, with augmentation and hyperparameter selection restricted to the training and validation partitions and the test partition reserved exclusively for final evaluation. On that locked test set, the confusion matrix contained 3,096 true negatives, 26 false positives, 29 false negatives, and 650 true positives. These counts correspond to 96.15% melanoma precision, 95.73% melanoma sensitivity, 99.17% specificity, a 95.94% F1-score, 97.45% balanced accuracy, and a Matthews correlation coefficient of 0.951. The reported ROC-AUC was 0.98.

The authors are careful to define the endpoint precisely: the positive class is melanoma and the negative class comprises all remaining ISIC 2019 diagnostic categories. That means the reported figures should not be read as benign-versus-malignant or general skin-cancer detection performance, since the negative pool includes other diagnostic categories rather than a simple benign label. This distinction matters for anyone interpreting the results in a clinical context, because the difficulty of a classification task depends heavily on how its classes are defined.

The authors also state that external validation, subgroup analysis, calibration, and expert validation of the model’s explanations remain necessary before any clinical use. The Grad-CAM++ visualizations are described as qualitative, and the study’s results come from a single benchmark dataset with a fixed holdout split rather than independent external cohorts. Within those limits, the study reports strong discrimination for the defined melanoma-versus-non-melanoma task and offers an example of how hybrid CNN-Transformer designs, lesion-aware preprocessing, and careful endpoint definition can be combined in dermoscopic image analysis.

Subject of Research: A hybrid CNN-Transformer deep learning model for classifying melanoma versus non-melanoma in dermoscopic images

Article Title: Design of a feature-optimized hybrid CNN–Transformer model for melanoma-vs.-non-melanoma classification

Article References: Ramya, R., & Jaisankar, N. (2026). Design of a feature-optimized hybrid CNN–Transformer model for melanoma-vs.-non-melanoma classification. Frontiers in Artificial Intelligence, 9, Article 1936676. https://doi.org/10.3389/frai.2026.1936676

Image Credits: AI Generated

DOI: 10.3389/frai.2026.1936676

Keywords: melanoma, dermoscopy, deep learning, CNN-Transformer, ISIC 2019, image classification, EfficientNet, Swin Transformer, Grad-CAM++, medical imaging, artificial intelligence, Design